In the following tests, we have measured the performance of several web application platforms, full-stack frameworks, and micro-frameworks (collectively, "frameworks"). For more information, read the introduction, motivation, and latest environment details.

2018-02-14

Round 15

A valentine for performance

Roses are red
Violets are blue
A fast server
Makes users love you

View additional commentary about Round 15 at our blog.

Showing 70 of 158 frameworks.

Filters

Classification

Disable all

Fullstack

Micro

Platform

Language

Enable all

C++

Clojure

Crystal

Dart

Elixir

Erlang

Groovy

Haskell

Java

JavaScript

Kotlin

Lua

Nim

Perl

PHP

Python

Ruby

Rust

Scala

Swift

Vala

Platform

Disable all

.NET

Akka

asyncio

Baratine

Bayou

blaze

Collie

Comsat

Cowboy

duda

ffead-cpp

GLib

JAX-RS

Jetty

JLHTTP

ktor

Light-4J

Meinheld

Netty

NIO2

nodejs

None

OpenResty

Plack

Pronghorn

Rack

Rapidoid

Ring

RingoJs

Rust

s-server

Servlet

Starlet

Tornado

Undertow

Ur/Web

uWSGI

Vert.x

vertx

Wai

Application operating system

Disable all

Linux

Windows

Front-end server

Disable all

actix-web

apache

blaze

ffead-cpp

Gunicorn

H2O

hyper

Hyper

Hypnotoad

IIS

Jetty

Meinheld

microhttpd

Monkey

nginx

nginx-uWSGI

None

Passenger

Puma

Resin

rouille

Soup

Starman

Thin

tokio-minihttp

Tornado

TorqueBox

Twisted Web

ULib

Undertow

Unicorn

uWSGI

Warp

Wildfly

workerman

Database-server

Disable all

MongoDB

MySQL

None

Postgres

SQLServer

Database operating system

Disable all

Linux

Windows

Object-relational mapper (ORM) classification

Disable all

Full

Micro

Raw

Implementation approach

Enable all

Realistic

Stripped

Framework

Disable all

aah

actframework

activeweb

actix

aiohttp

akka-http

aleph

Amber

API Star

api_hour

ASP.NET

ASP.NET Core

beego

blade

blaze

bottle

cakephp

chicagoboss

clancatsframework

codeigniter

Collie

colossus

compojure

cpoll-cppsp

curacao

cutelyst

cygnite

dancer

django

dropwizard

evhttp-sharp

express

facil.io

falcon

falcore

Fat Free

ffead-cpp

finagle

finatra

finch

fintrospect

flask

fuel

gemini

gin

Goji

grails

grape

grizzly

hanami

hapi

Hexagon

hhvm

Hot

http4k

http4s

hunt

hyper

iron

japronto

jawn

Jersey

jester

jooby

kami

kelp

kemal

Klein

koa

kohana

ktor

lapis

laravel

Limonade

lithium

Lumen

luminus

luna

macchiato

Minijax

mojolicious

morepath

nancy

netty

ngx_mruby

nickel

ninja

None

Octopus

openresty

padrino

pedestal

phalcon

Phoenix

phpixie

phreeze

plack

play1

play2

Proteus

pyramid

rack

rack-sequel

rails

rapidoid

rapidoid-http-fast

redstone

reitit

rest-express

Revenj

Revenj.JVM

ringo

roda-sequel

rouille

s-server

sanic

scruffy

Servant

servicestack

silex

silicon

sinatra

sinatra-sequel

slim

snap

spark

Spock

spray

spring

start

stream

symfony

tapestry

tokio-minihttp

treefrog

TurboGears

ULib

urweb

uvicorn

Valum

Vapor

vertx-web

vibe.d

vibed

VSGI

web-simple

web2py

webgo

Webware

weppy

wheezy.web

wicket

yaf

yesod

yii2

Zend

ZendFramework1

Key

Enabled

Disabled

Unavailable

Test types

Hardware

JSON serialization

Single query

Multiple queries

Fortunes

Data updates

Plaintext

Physical

Cloud

JSON serialization

Best database-access responses per second, single query, Dell servers at ServerCentral(152 tests)

Framework	Best performance (higher is better)		Cls	Lng	Plt	FE	Aos	DB	Dos	Orm	IA	Errors
vertx-postgres	191,353	100.0%	Plt	Jav	Ver	Non	Lin	Pg	Lin	Raw	Rea	0
ulib-postgres_fit	187,519	98.0%	Plt	C++	Non	ULi	Lin	Pg	Lin	Mcr	Rea	0
ulib-postgres	182,112	95.2%	Plt	C++	Non	ULi	Lin	Pg	Lin	Mcr	Rea	0
vertx-web-postgres	176,393	92.2%	Mcr	Jav	vtx	Non	Lin	Pg	Lin	Raw	Rea	0
ulib-mongodb	175,101	91.5%	Plt	C++	Non	ULi	Lin	Mo	Lin	Mcr	Rea	0
proteus	175,048	91.5%	Mcr	Jav	Utw	Non	Lin	Pg	Lin	Raw	Rea	0
gemini-postgres	174,856	91.4%	Ful	Jav	Svt	Res	Lin	Pg	Lin	Mcr	Rea	0
revenj-jvm	172,092	89.9%	Ful	Jav	Svt	Res	Lin	Pg	Lin	Ful	Rea	0
light-4j-postgresql	171,838	89.8%	Plt	Jav	Lig	Non	Lin	Pg	Lin	Raw	Rea	0
undertow-postgresql	166,597	87.1%	Plt	Jav	Utw	Non	Lin	Pg	Lin	Raw	Rea	0
cpoll_cppsp-postgres	165,134	86.3%	Plt	C++	Non	Non	Lin	Pg	Lin	Raw	Rea	0
cpoll_cppsp-postgres	160,247	83.7%	Plt	C++	Non	Non	Lin	Pg	Lin	Raw	Rea	0
cutelyst-thread-post	156,352	81.7%	Ful	C++	Qt	Non	Lin	Pg	Lin	Raw	Rea	0
cutelyst-thread-post	154,719	80.9%	Ful	C++	Qt	Non	Lin	Pg	Lin	Raw	Rea	0
ulib-mysql	149,247	78.0%	Plt	C++	Non	ULi	Lin	My	Lin	Mcr	Rea	0
cpoll_cppsp-raw	148,204	77.5%	Plt	C++	Non	Non	Lin	My	Lin	Raw	Rea	0
fasthttp-postgresql	137,073	71.6%	Plt	Go	Non	Non	Lin	Pg	Lin	Raw	Rea	0
undertow-mysql	136,282	71.2%	Plt	Jav	Utw	Non	Lin	My	Lin	Raw	Rea	0
proteus-mysql	134,560	70.3%	Mcr	Jav	Utw	Non	Lin	My	Lin	Raw	Rea	0
jawn	129,041	67.4%	Ful	Jav	Svt	Utw	Lin	Pg	Lin	Raw	Rea	0
undertow-mongodb	128,561	67.2%	Plt	Jav	Utw	Non	Lin	Mo	Lin	Raw	Rea	0
go-prefork	124,834	65.2%	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
cutelyst-pf-postgres	117,714	61.5%	Ful	C++	Qt	Non	Lin	Pg	Lin	Raw	Rea	0
vertx-web-susom-post	113,360	59.2%	Mcr	Jav	vtx	Non	Lin	Pg	Lin	Raw	Rea	0
gemini-mysql	113,041	59.1%	Ful	Jav	Svt	Res	Lin	My	Lin	Mcr	Rea	0
cutelyst-thread-mysq	107,164	56.0%	Ful	C++	Qt	Non	Lin	My	Lin	Raw	Rea	0
cutelyst-thread-mysq	106,255	55.5%	Ful	C++	Qt	Non	Lin	My	Lin	Raw	Rea	0
cutelyst-nginx-postg	105,743	55.3%	Ful	C++	Qt	ngx	Lin	Pg	Lin	Raw	Rea	0
kami	101,111	52.8%	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
go-interpolate	98,849	51.7%	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
fintrospect	96,913	50.6%	Mcr	Sca	Nty	Non	Lin	My	Lin	Raw	Rea	0
actframework-ebean-p	96,473	50.4%	Ful	Jav	Utw	Non	Lin	Pg	Lin	Ful	Rea	0
jooby	94,281	49.3%	Ful	Jav	Nty	Non	Lin	Pg	Lin	Raw	Rea	0
actframework-morphia	93,612	48.9%	Ful	Jav	Utw	Non	Lin	Mo	Lin	Ful	Rea	0
actframework-hiberna	93,110	48.7%	Ful	Jav	Utw	Non	Lin	Pg	Lin	Ful	Rea	0
akka-http	87,715	45.8%	Mcr	Sca	Akk	Non	Lin	My	Lin	Raw	Rea	0
cutelyst-pf-mysql	83,682	43.7%	Ful	C++	Qt	Non	Lin	My	Lin	Raw	Rea	0
cutelyst-nginx-mysql	79,649	41.6%	Ful	C++	Qt	ngx	Lin	My	Lin	Raw	Rea	0
actframework-ebean-m	79,635	41.6%	Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	0
bottle-mysql-raw	77,745	40.6%	Mcr	Py	Mei	Non	Lin	My	Lin	Raw	Rea	0
grizzly-jersey	73,961	38.7%	Mcr	Jav	JAX	Non	Lin	My	Lin	Ful	Rea	0
treefrog-mongodb	73,047	38.2%	Ful	C++	Non	Non	Lin	Mo	Lin	Mcr	Rea	0
actframework-hiberna	70,192	36.7%	Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	0
silicon	68,945	36.0%	Mcr	C++	Non	mic	Lin	My	Lin	Ful	Rea	0
http4s	66,460	34.7%	Mcr	Sca	NIO	bla	Lin	Pg	Lin	Raw	Rea	0
fasthttp	66,424	34.7%	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
servlet-raw	63,712	33.3%	Plt	Jav	Svt	Res	Lin	My	Lin	Raw	Rea	0
cutelyst-pf-mysql-ep	63,613	33.2%	Ful	C++	Qt	Non	Lin	My	Lin	Raw	Rea	0
wildfly-ee7	62,868	32.9%	Ful	Jav	Svt	Wil	Lin	My	Lin	Ful	Rea	0
falcore	60,414	31.6%	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
goji	59,755	31.2%	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
wt-postgres	59,551	31.1%	Ful	C++	Non	Non	Lin	Pg	Lin	Ful	Rea	0
beego-orm-mysql	59,302	31.0%	Ful	Go	Non	Non	Lin	My	Lin	Mcr	Rea	0
go-postgres	59,208	30.9%	Plt	Go	Non	Non	Lin	Pg	Lin	Raw	Rea	0
treefrog	59,000	30.8%	Ful	C++	Non	Non	Lin	My	Lin	Mcr	Rea	0
go	58,969	30.8%	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
gin	58,883	30.8%	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
morepath	58,150	30.4%	Mcr	Py	Mei	Gun	Lin	Pg	Lin	Ful	Rea	0
servlet-postgres-raw	57,211	29.9%	Plt	Jav	Svt	Res	Lin	Pg	Lin	Raw	Rea	0
vertx-web-mongodb	56,797	29.7%	Mcr	Jav	vtx	Non	Lin	Mo	Lin	Raw	Rea	0
dropwizard-mysql	56,521	29.5%	Ful	Jav	JAX	Jty	Lin	My	Lin	Ful	Rea	0
wt	56,013	29.3%	Ful	C++	Non	Non	Lin	My	Lin	Ful	Rea	0
undertow-jersey-hika	55,937	29.2%	Plt	Jav	JAX	Non	Lin	My	Lin	Ful	Rea	0
revel-raw	55,320	28.9%	Ful	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
dropwizard-postgres	55,163	28.8%	Ful	Jav	JAX	Jty	Lin	Pg	Lin	Ful	Rea	0
undertow-mongodb-asy	54,970	28.7%	Plt	Jav	Utw	Non	Lin	Mo	Lin	Raw	Rea	0
echo	54,578	28.5%	Mcr	Go	Non	Non	Lin	Pg	Lin	Raw	Rea	0
revel-jet	53,746	28.1%	Ful	Go	Non	Non	Lin	My	Lin	Mcr	Rea	0
revel-qbs	53,019	27.7%	Ful	Go	Non	Non	Lin	My	Lin	Mcr	Rea	0
treefrog-hybrid	52,684	27.5%	Ful	C++	Non	Non	Lin	My	Lin	Mcr	Rea	2
treefrog-postgres	49,012	25.6%	Ful	C++	Non	Non	Lin	Pg	Lin	Mcr	Rea	0
play2-scala-anorm-li	48,890	25.5%	Ful	Sca	Nty	Non	Lin	My	Lin	Ful	Rea	0
tornado-pypy2	47,638	24.9%	Plt	Py	Non	Tor	Lin	Mo	Lin	Raw	Rea	0
activeweb	46,120	24.1%	Ful	Jav	Svt	Non	Lin	My	Lin	Mcr	Rea	0
activeweb-jackson	46,045	24.1%	Ful	Jav	Svt	Non	Lin	My	Lin	Mcr	Rea	0
weppy-py3	46,018	24.0%	Ful	Py	Mei	Non	Lin	Pg	Lin	Ful	Rea	0
flask-mysql-raw	45,601	23.8%	Mcr	Py	Mei	Non	Lin	My	Lin	Raw	Rea	0
api_hour-mysql	44,877	23.5%	Mcr	Py	asy	Gun	Lin	My	Lin	Raw	Rea	0
minijax	44,028	23.0%	Ful	Jav	JAX	Jty	Lin	My	Lin	Ful	Rea	0
api_hour	42,563	22.2%	Mcr	Py	asy	Gun	Lin	Pg	Lin	Raw	Rea	0
weppy	42,099	22.0%	Ful	Py	Mei	Non	Lin	Pg	Lin	Ful	Rea	0
play1	41,635	21.8%	Ful	Jav	Nty	Non	Lin	My	Lin	Ful	Rea	0
restexpress	40,243	21.0%	Mcr	Jav	Nty	Non	Lin	Mo	Lin	Raw	Rea	0
aiohttp	40,097	21.0%	Mcr	Py	asy	Gun	Lin	Pg	Lin	Ful	Rea	0
play2-scala-anorm-li	38,457	20.1%	Ful	Sca	Akk	Non	Lin	My	Lin	Ful	Rea	0
dropwizard-jdbi-post	36,582	19.1%	Ful	Jav	JAX	Jty	Lin	Pg	Lin	Mcr	Rea	0
play2-java-jpa-hikar	36,043	18.8%	Ful	Jav	Nty	Non	Lin	My	Lin	Ful	Rea	0
aiohttp-pg-raw	35,968	18.8%	Mcr	Py	asy	Gun	Lin	Pg	Lin	Raw	Rea	0
wicket	35,055	18.3%	Ful	Jav	Svt	Res	Lin	My	Lin	Ful	Rea	0
play2-java-jpa-hikar	34,686	18.1%	Ful	Jav	Akk	Non	Lin	My	Lin	Ful	Rea	0
dropwizard-mongodb	32,794	17.1%	Ful	Jav	JAX	Jty	Lin	Mo	Lin	Ful	Rea	0
tapestry	32,316	16.9%	Ful	Jav	Svt	Res	Lin	My	Lin	Ful	Rea	0
aah	31,799	16.6%	Ful	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
undertow-jersey-c3p0	31,657	16.5%	Plt	Jav	JAX	Non	Lin	My	Lin	Ful	Rea	0
django-py3	31,229	16.3%	Ful	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
pyramid	28,486	14.9%	Ful	Py	Non	Mei	Lin	Pg	Lin	Ful	Rea	0
pyramid-py2	28,184	14.7%	Ful	Py	Non	Mei	Lin	Pg	Lin	Ful	Rea	0
wheezyweb	27,585	14.4%	Mcr	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
spark	25,911	13.5%	Mcr	Jav	Svt	Res	Lin	My	Lin	Ful	Rea	0
play2-java-jooq-hika	25,736	13.4%	Ful	Jav	Nty	Non	Lin	My	Lin	Ful	Rea	0
play2-java-jooq-hika	24,532	12.8%	Ful	Jav	Akk	Non	Lin	My	Lin	Ful	Rea	0
django-postgresql	23,981	12.5%	Ful	Py	Non	Mei	Lin	Pg	Lin	Ful	Rea	0
scruffy	22,867	12.0%	Mcr	Sca	Utw	Non	Lin	Mo	Lin	Raw	Rea	54,214
django	22,507	11.8%	Ful	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
blade	22,100	11.5%	Ful	Jav	Non	Non	Lin	My	Lin	Ful	Rea	0
play2-java-ebean-hik	22,022	11.5%	Ful	Jav	Nty	Non	Lin	My	Lin	Ful	Rea	0
flask-py3	21,640	11.3%	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
uvicorn	21,446	11.2%	Plt	Py	Non	Non	Lin	Pg	Lin	Raw	Rea	0
bottle-py3	21,066	11.0%	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
bottle	20,751	10.8%	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
flask	20,639	10.8%	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
play2-java-ebean-hik	20,368	10.6%	Ful	Jav	Akk	Non	Lin	My	Lin	Ful	Rea	0
turbogears	18,908	9.9%	Mcr	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
tornado	17,015	8.9%	Plt	Py	Non	Tor	Lin	Mo	Lin	Raw	Rea	0
tornado-py3	16,362	8.6%	Plt	Py	Non	Tor	Lin	Mo	Lin	Raw	Rea	0
ffead-cpp-apache-mon	15,902	8.3%	Ful	C++	ffe	Apc	Lin	Mo	Lin	Ful	Rea	0
ffead-cpp	15,834	8.3%	Ful	C++	ffe	ffe	Lin	Mo	Lin	Ful	Rea	0
ffead-cpp-nginx-mong	15,824	8.3%	Ful	C++	ffe	ngx	Lin	Mo	Lin	Ful	Rea	0
go-mongo	15,802	8.3%	Plt	Go	Non	Non	Lin	Mo	Lin	Raw	Rea	0
cppcms-postgres	15,751	8.2%	Plt	C++	Non	Non	Lin	Pg	Lin	Raw	Rea	0
spring	15,130	7.9%	Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	0
play2-scala-slick	14,990	7.8%	Ful	Sca	Akk	Non	Lin	My	Lin	Ful	Rea	0
cppcms	14,820	7.7%	Plt	C++	Non	Non	Lin	My	Lin	Raw	Rea	0
play2-scala-slick-ne	13,871	7.2%	Ful	Sca	Nty	Non	Lin	My	Lin	Ful	Rea	0
ffead-cpp-mysql	12,559	6.6%	Ful	C++	ffe	ffe	Lin	My	Lin	Ful	Rea	0
ffead-cpp-postgresql	12,105	6.3%	Ful	C++	ffe	ffe	Lin	Pg	Lin	Ful	Rea	0
ffead-cpp-nginx-post	12,077	6.3%	Ful	C++	ffe	ngx	Lin	Pg	Lin	Ful	Rea	0
ffead-cpp-apache-pos	12,034	6.3%	Ful	C++	ffe	Apc	Lin	Pg	Lin	Ful	Rea	0
web2py-optimized	11,435	6.0%	Ful	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
cppcms-direct	10,962	5.7%	Plt	C++	Non	Non	Lin	My	Lin	Raw	Rea	0
ninja-standalone	8,621	4.5%	Ful	Jav	Jty	Non	Lin	My	Lin	Ful	Rea	0
web2py	8,587	4.5%	Ful	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
tornado-postgresql-r	8,176	4.3%	Plt	Py	Non	Tor	Lin	Pg	Lin	Raw	Rea	0
bottle-nginx-uwsgi	5,066	2.6%	Mcr	Py	uWS	ngx	Lin	My	Lin	Ful	Rea	0
weppy-nginx-uwsgi	4,508	2.4%	Ful	Py	uWS	ngx	Lin	Pg	Lin	Ful	Rea	0
flask-nginx-uwsgi	4,462	2.3%	Mcr	Py	Non	ngx	Lin	My	Lin	Ful	Rea	0
flask-pypy2-mysql-ra	4,202	2.2%	Mcr	Py	Tor	Non	Lin	My	Lin	Raw	Rea	0
weppy-pypy2	3,496	1.8%	Ful	Py	Tor	Non	Lin	Pg	Lin	Ful	Rea	0
bottle-pypy2	2,209	1.2%	Mcr	Py	Tor	Non	Lin	My	Lin	Ful	Rea	0
play2-scala-reactive	1,894	1.0%	Ful	Sca	Akk	Non	Lin	Mo	Lin	Ful	Rea	0
play2-scala-reactive	1,772	0.9%	Ful	Sca	Nty	Non	Lin	Mo	Lin	Ful	Rea	0
flask-pypy2	1,713	0.9%	Mcr	Py	Non	Tor	Lin	My	Lin	Ful	Rea	0
restexpress-mysql-ra	660	0.3%	Mcr	Jav	Nty	Non	Lin	My	Lin	Raw	Rea	121,507
klein	323	0.2%	Mcr	Py	Non	Twi	Lin	My	Lin	Ful	Rea	0
cherrypy-py3	285	0.1%	Mcr	Py	Non	Non	Lin	My	Lin	Ful	Rea	0
cherrypy	195	0.1%	Mcr	Py	Non	Non	Lin	My	Lin	Ful	Rea	319
ffead-cpp-apache-mys	0	0.0%	Ful	C++	ffe	Apc	Lin	My	Lin	Ful	Rea	0
ffead-cpp-nginx-mysq	0	0.0%	Ful	C++	ffe	ngx	Lin	My	Lin	Ful	Rea	0
webware	0	0.0%	Mcr	Py	Non	Non	Lin	My	Lin	Ful	Rea	1,143
actframework-eclipse	—	Did not complete	Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	—
actframework-eclipse	—	Did not complete	Ful	Jav	Utw	Non	Lin	Pg	Lin	Ful	Rea	—
wheezyweb-py3	—	Did not complete	Mcr	Py	Non	Mei	Lin	My	Lin	Ful	Rea	—

Database-access responses per second, single query, Dell servers at ServerCentral

Framework	16	32	64	128	256	512	Best	Cls	Lng	Plt	FE	Aos	DB	Dos	Orm	IA	Errors
aah	20,974	22,414	24,019	27,994	30,620	31,799	31,799	Ful	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
actframework-ebean-m	27,127	43,985	65,300	79,635	79,321	79,157	79,635	Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	0
actframework-ebean-p	30,073	44,924	74,314	96,473	89,504	90,890	96,473	Ful	Jav	Utw	Non	Lin	Pg	Lin	Ful	Rea	0
actframework-hiberna	22,093	43,084	60,106	69,449	70,192	69,484	70,192	Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	0
actframework-hiberna	27,279	46,671	73,541	93,110	88,908	89,553	93,110	Ful	Jav	Utw	Non	Lin	Pg	Lin	Ful	Rea	0
actframework-morphia	23,930	36,369	70,110	90,932	93,612	92,495	93,612	Ful	Jav	Utw	Non	Lin	Mo	Lin	Ful	Rea	0
actframework-eclipse	— Did not complete —							Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	—
actframework-eclipse	— Did not complete —							Ful	Jav	Utw	Non	Lin	Pg	Lin	Ful	Rea	—
activeweb	17,511	32,550	40,577	42,977	44,345	46,120	46,120	Ful	Jav	Svt	Non	Lin	My	Lin	Mcr	Rea	0
activeweb-jackson	17,964	33,037	40,698	43,437	43,986	46,045	46,045	Ful	Jav	Svt	Non	Lin	My	Lin	Mcr	Rea	0
aiohttp	8,600	17,510	30,062	34,725	40,097	39,666	40,097	Mcr	Py	asy	Gun	Lin	Pg	Lin	Ful	Rea	0
aiohttp-pg-raw	10,628	21,854	35,149	35,968	34,138	33,149	35,968	Mcr	Py	asy	Gun	Lin	Pg	Lin	Raw	Rea	0
akka-http	17,513	23,099	48,503	87,715	85,615	86,524	87,715	Mcr	Sca	Akk	Non	Lin	My	Lin	Raw	Rea	0
api_hour	10,566	17,737	33,325	39,544	42,563	35,043	42,563	Mcr	Py	asy	Gun	Lin	Pg	Lin	Raw	Rea	0
api_hour-mysql	9,689	17,812	30,629	36,078	42,098	44,877	44,877	Mcr	Py	asy	Gun	Lin	My	Lin	Raw	Rea	0
beego-orm-mysql	19,002	27,869	47,251	57,795	59,302	42,854	59,302	Ful	Go	Non	Non	Lin	My	Lin	Mcr	Rea	0
blade	21,495	21,302	21,528	21,839	22,100	21,107	22,100	Ful	Jav	Non	Non	Lin	My	Lin	Ful	Rea	0
bottle	6,585	12,066	20,751	19,393	16,992	16,915	20,751	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
bottle-mysql-raw	19,206	37,458	66,936	76,322	77,745	70,177	77,745	Mcr	Py	Mei	Non	Lin	My	Lin	Raw	Rea	0
bottle-nginx-uwsgi	4,202	4,292	4,537	5,066	4,319	4,271	5,066	Mcr	Py	uWS	ngx	Lin	My	Lin	Ful	Rea	0
bottle-py3	6,326	12,167	21,066	20,375	18,229	18,102	21,066	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
bottle-pypy2	1,480	1,770	2,209	1,888	1,467	1,215	2,209	Mcr	Py	Tor	Non	Lin	My	Lin	Ful	Rea	0
cpoll_cppsp-postgres	46,931	111,971	160,247	153,056	117,891	96,565	160,247	Plt	C++	Non	Non	Lin	Pg	Lin	Raw	Rea	0
cpoll_cppsp-postgres	41,328	106,961	165,134	127,400	155,022	127,968	165,134	Plt	C++	Non	Non	Lin	Pg	Lin	Raw	Rea	0
cpoll_cppsp-raw	40,605	97,389	144,929	148,204	144,076	130,261	148,204	Plt	C++	Non	Non	Lin	My	Lin	Raw	Rea	0
cutelyst-nginx-mysql	19,678	46,430	79,523	79,649	79,454	79,103	79,649	Ful	C++	Qt	ngx	Lin	My	Lin	Raw	Rea	0
cutelyst-nginx-postg	23,877	49,670	105,743	82,242	80,978	80,861	105,743	Ful	C++	Qt	ngx	Lin	Pg	Lin	Raw	Rea	0
cutelyst-pf-mysql	10,493	18,950	49,268	62,009	82,739	83,682	83,682	Ful	C++	Qt	Non	Lin	My	Lin	Raw	Rea	0
cutelyst-pf-mysql-ep	12,706	23,402	34,679	60,192	63,613	53,774	63,613	Ful	C++	Qt	Non	Lin	My	Lin	Raw	Rea	0
cutelyst-pf-postgres	11,517	20,821	32,106	65,950	98,344	117,714	117,714	Ful	C++	Qt	Non	Lin	Pg	Lin	Raw	Rea	0
cutelyst-thread-mysq	22,670	59,071	84,070	101,966	106,336	107,164	107,164	Ful	C++	Qt	Non	Lin	My	Lin	Raw	Rea	0
cutelyst-thread-mysq	26,091	60,025	84,368	102,096	105,678	106,255	106,255	Ful	C++	Qt	Non	Lin	My	Lin	Raw	Rea	0
cutelyst-thread-post	32,543	76,726	121,469	149,807	154,719	153,972	154,719	Ful	C++	Qt	Non	Lin	Pg	Lin	Raw	Rea	0
cutelyst-thread-post	32,291	64,780	112,416	154,012	156,352	155,621	156,352	Ful	C++	Qt	Non	Lin	Pg	Lin	Raw	Rea	0
django	7,687	14,618	22,507	22,201	17,522	17,992	22,507	Ful	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
django-postgresql	7,842	15,092	23,981	23,312	19,263	18,696	23,981	Ful	Py	Non	Mei	Lin	Pg	Lin	Ful	Rea	0
django-py3	9,163	18,277	30,761	31,229	26,699	26,337	31,229	Ful	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
dropwizard-jdbi-post	21,376	30,745	33,227	36,259	36,337	36,582	36,582	Ful	Jav	JAX	Jty	Lin	Pg	Lin	Mcr	Rea	0
dropwizard-mongodb	20,234	30,474	31,727	32,794	32,272	31,682	32,794	Ful	Jav	JAX	Jty	Lin	Mo	Lin	Ful	Rea	0
dropwizard-mysql	20,839	37,272	56,521	51,795	47,310	49,602	56,521	Ful	Jav	JAX	Jty	Lin	My	Lin	Ful	Rea	0
dropwizard-postgres	23,188	40,454	55,163	51,961	45,745	47,103	55,163	Ful	Jav	JAX	Jty	Lin	Pg	Lin	Ful	Rea	0
falcore	29,022	40,630	60,414	45,103	45,533	46,599	60,414	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
ffead-cpp	14,751	15,604	15,834	15,531	15,521	15,249	15,834	Ful	C++	ffe	ffe	Lin	Mo	Lin	Ful	Rea	0
ffead-cpp-apache-mon	14,693	15,902	15,864	15,644	15,589	15,308	15,902	Ful	C++	ffe	Apc	Lin	Mo	Lin	Ful	Rea	0
ffead-cpp-apache-mys	0	0	0	0	0	0	0	Ful	C++	ffe	Apc	Lin	My	Lin	Ful	Rea	0
ffead-cpp-apache-pos	9,976	12,034	7,558	6,669	6,608	6,602	12,034	Ful	C++	ffe	Apc	Lin	Pg	Lin	Ful	Rea	0
ffead-cpp-mysql	10,435	12,559	0	0	0	0	12,559	Ful	C++	ffe	ffe	Lin	My	Lin	Ful	Rea	0
ffead-cpp-nginx-mong	14,673	15,824	15,810	15,643	15,614	15,761	15,824	Ful	C++	ffe	ngx	Lin	Mo	Lin	Ful	Rea	0
ffead-cpp-nginx-mysq	0	0	0	0	0	0	0	Ful	C++	ffe	ngx	Lin	My	Lin	Ful	Rea	0
ffead-cpp-nginx-post	9,956	12,077	7,645	6,793	6,650	6,641	12,077	Ful	C++	ffe	ngx	Lin	Pg	Lin	Ful	Rea	0
ffead-cpp-postgresql	9,944	12,105	7,640	6,682	6,645	6,639	12,105	Ful	C++	ffe	ffe	Lin	Pg	Lin	Ful	Rea	0
fintrospect	22,348	44,456	85,385	95,887	96,913	95,621	96,913	Mcr	Sca	Nty	Non	Lin	My	Lin	Raw	Rea	0
flask	6,762	12,901	20,639	18,135	16,180	15,628	20,639	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
flask-mysql-raw	12,312	23,543	40,858	45,601	36,068	35,476	45,601	Mcr	Py	Mei	Non	Lin	My	Lin	Raw	Rea	0
flask-nginx-uwsgi	4,099	4,172	4,462	4,439	4,272	4,201	4,462	Mcr	Py	Non	ngx	Lin	My	Lin	Ful	Rea	0
flask-py3	6,788	13,101	21,640	20,943	18,301	18,349	21,640	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
flask-pypy2	1,245	1,667	1,713	1,433	1,048	957	1,713	Mcr	Py	Non	Tor	Lin	My	Lin	Ful	Rea	0
flask-pypy2-mysql-ra	2,620	2,961	4,202	3,277	2,402	2,270	4,202	Mcr	Py	Tor	Non	Lin	My	Lin	Raw	Rea	0
gemini-mysql	34,151	74,187	108,185	113,041	109,097	63,432	113,041	Ful	Jav	Svt	Res	Lin	My	Lin	Mcr	Rea	0
gemini-postgres	40,426	103,505	167,746	174,856	150,425	126,070	174,856	Ful	Jav	Svt	Res	Lin	Pg	Lin	Mcr	Rea	0
gin	27,955	39,923	58,883	46,910	47,168	48,555	58,883	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
goji	25,112	40,169	59,755	41,211	25,763	30,180	59,755	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
grizzly-jersey	18,315	38,780	58,828	72,758	73,717	73,961	73,961	Mcr	Jav	JAX	Non	Lin	My	Lin	Ful	Rea	0
http4s	18,178	39,729	64,159	66,460	65,411	65,770	66,460	Mcr	Sca	NIO	bla	Lin	Pg	Lin	Raw	Rea	0
jawn	33,707	44,377	48,105	128,935	129,041	125,071	129,041	Ful	Jav	Svt	Utw	Lin	Pg	Lin	Raw	Rea	0
undertow-jersey-c3p0	9,427	21,166	31,657	27,816	25,277	25,567	31,657	Plt	Jav	JAX	Non	Lin	My	Lin	Ful	Rea	0
undertow-jersey-hika	16,561	33,905	43,897	55,937	51,441	50,831	55,937	Plt	Jav	JAX	Non	Lin	My	Lin	Ful	Rea	0
jooby	34,270	59,839	85,202	91,466	81,710	94,281	94,281	Ful	Jav	Nty	Non	Lin	Pg	Lin	Raw	Rea	0
kami	27,737	39,957	71,522	101,111	92,695	88,222	101,111	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
klein	322	323	323	322	315	314	323	Mcr	Py	Non	Twi	Lin	My	Lin	Ful	Rea	0
minijax	30,576	38,436	41,373	43,542	44,028	43,855	44,028	Ful	Jav	JAX	Jty	Lin	My	Lin	Ful	Rea	0
morepath	12,441	24,600	47,972	58,150	54,867	48,981	58,150	Mcr	Py	Mei	Gun	Lin	Pg	Lin	Ful	Rea	0
ninja-standalone	8,389	8,531	8,347	8,492	8,621	8,573	8,621	Ful	Jav	Jty	Non	Lin	My	Lin	Ful	Rea	0
cherrypy	186	194	192	191	193	195	195	Mcr	Py	Non	Non	Lin	My	Lin	Ful	Rea	319
cherrypy-py3	285	285	285	283	284	282	285	Mcr	Py	Non	Non	Lin	My	Lin	Ful	Rea	0
cppcms	14,175	14,820	14,535	14,146	14,292	14,456	14,820	Plt	C++	Non	Non	Lin	My	Lin	Raw	Rea	0
cppcms-direct	10,470	10,641	10,796	10,962	10,683	10,826	10,962	Plt	C++	Non	Non	Lin	My	Lin	Raw	Rea	0
cppcms-postgres	15,751	15,698	15,099	14,555	14,535	14,457	15,751	Plt	C++	Non	Non	Lin	Pg	Lin	Raw	Rea	0
echo	25,894	42,037	54,578	44,193	43,631	37,639	54,578	Mcr	Go	Non	Non	Lin	Pg	Lin	Raw	Rea	0
fasthttp	34,832	43,077	66,424	51,097	48,464	38,234	66,424	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
fasthttp-postgresql	35,255	73,183	81,311	103,468	137,073	90,577	137,073	Plt	Go	Non	Non	Lin	Pg	Lin	Raw	Rea	0
go	28,488	40,564	58,969	48,303	48,058	36,924	58,969	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
go-interpolate	28,400	40,149	70,194	98,042	98,849	52,891	98,849	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
go-mongo	15,740	15,802	15,681	15,569	15,512	15,262	15,802	Plt	Go	Non	Non	Lin	Mo	Lin	Raw	Rea	0
go-postgres	27,256	41,539	59,208	47,350	46,694	38,860	59,208	Plt	Go	Non	Non	Lin	Pg	Lin	Raw	Rea	0
go-prefork	33,048	74,450	110,412	121,515	124,834	124,341	124,834	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
light-4j-postgresql	40,329	55,670	114,170	171,838	170,944	171,215	171,838	Plt	Jav	Lig	Non	Lin	Pg	Lin	Raw	Rea	0
revel-jet	22,906	35,072	53,746	43,417	44,092	45,022	53,746	Ful	Go	Non	Non	Lin	My	Lin	Mcr	Rea	0
revel-qbs	23,251	36,149	53,019	43,664	44,237	45,821	53,019	Ful	Go	Non	Non	Lin	My	Lin	Mcr	Rea	0
revel-raw	23,848	37,710	55,320	44,963	45,171	46,849	55,320	Ful	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
servlet-postgres-raw	35,780	52,041	52,859	53,765	57,211	52,766	57,211	Plt	Jav	Svt	Res	Lin	Pg	Lin	Raw	Rea	0
servlet-raw	31,479	52,014	52,644	53,169	58,644	63,712	63,712	Plt	Jav	Svt	Res	Lin	My	Lin	Raw	Rea	0
tornado	5,879	10,375	16,201	17,015	16,098	15,493	17,015	Plt	Py	Non	Tor	Lin	Mo	Lin	Raw	Rea	0
tornado-postgresql-r	6,865	7,632	7,917	8,176	8,008	7,963	8,176	Plt	Py	Non	Tor	Lin	Pg	Lin	Raw	Rea	0
tornado-py3	4,990	9,313	13,904	15,285	16,362	16,183	16,362	Plt	Py	Non	Tor	Lin	Mo	Lin	Raw	Rea	0
tornado-pypy2	5,190	10,788	20,966	29,762	44,255	47,638	47,638	Plt	Py	Non	Tor	Lin	Mo	Lin	Raw	Rea	0
undertow-mongodb	29,611	42,798	64,437	125,942	128,561	88,554	128,561	Plt	Jav	Utw	Non	Lin	Mo	Lin	Raw	Rea	0
undertow-mongodb-asy	30,212	50,268	48,299	50,984	54,970	53,978	54,970	Plt	Jav	Utw	Non	Lin	Mo	Lin	Raw	Rea	0
undertow-mysql	34,159	47,697	75,837	129,063	136,282	133,451	136,282	Plt	Jav	Utw	Non	Lin	My	Lin	Raw	Rea	0
undertow-postgresql	38,375	47,162	77,333	160,148	165,171	166,597	166,597	Plt	Jav	Utw	Non	Lin	Pg	Lin	Raw	Rea	0
vertx-postgres	38,074	105,005	168,967	178,666	183,391	191,353	191,353	Plt	Jav	Ver	Non	Lin	Pg	Lin	Raw	Rea	0
wildfly-ee7	16,141	21,838	26,873	47,102	53,407	62,868	62,868	Ful	Jav	Svt	Wil	Lin	My	Lin	Ful	Rea	0
play1	14,772	27,663	39,354	41,635	41,420	41,187	41,635	Ful	Jav	Nty	Non	Lin	My	Lin	Ful	Rea	0
play2-java-ebean-hik	12,035	19,048	19,876	20,057	20,176	20,368	20,368	Ful	Jav	Akk	Non	Lin	My	Lin	Ful	Rea	0
play2-java-ebean-hik	13,466	22,022	21,546	21,579	21,854	21,266	22,022	Ful	Jav	Nty	Non	Lin	My	Lin	Ful	Rea	0
play2-java-jooq-hika	14,995	22,463	21,066	24,367	23,282	24,532	24,532	Ful	Jav	Akk	Non	Lin	My	Lin	Ful	Rea	0
play2-java-jooq-hika	17,056	25,717	24,423	25,736	22,968	23,185	25,736	Ful	Jav	Nty	Non	Lin	My	Lin	Ful	Rea	0
play2-java-jpa-hikar	19,245	34,196	34,053	34,296	34,276	34,686	34,686	Ful	Jav	Akk	Non	Lin	My	Lin	Ful	Rea	0
play2-java-jpa-hikar	22,289	36,002	36,043	35,904	35,189	34,718	36,043	Ful	Jav	Nty	Non	Lin	My	Lin	Ful	Rea	0
play2-scala-anorm-li	22,534	34,410	37,150	37,921	38,456	38,457	38,457	Ful	Sca	Akk	Non	Lin	My	Lin	Ful	Rea	0
play2-scala-anorm-li	26,365	47,000	46,663	48,890	46,363	47,179	48,890	Ful	Sca	Nty	Non	Lin	My	Lin	Ful	Rea	0
play2-scala-reactive	1,694	1,807	1,884	1,894	1,885	1,858	1,894	Ful	Sca	Akk	Non	Lin	Mo	Lin	Ful	Rea	0
play2-scala-reactive	1,586	1,713	1,718	1,750	1,751	1,772	1,772	Ful	Sca	Nty	Non	Lin	Mo	Lin	Ful	Rea	0
play2-scala-slick	11,994	14,990	13,726	14,016	13,859	13,594	14,990	Ful	Sca	Akk	Non	Lin	My	Lin	Ful	Rea	0
play2-scala-slick-ne	12,983	13,350	13,871	13,485	13,504	13,547	13,871	Ful	Sca	Nty	Non	Lin	My	Lin	Ful	Rea	0
proteus	42,346	58,960	119,667	175,048	173,354	173,329	175,048	Mcr	Jav	Utw	Non	Lin	Pg	Lin	Raw	Rea	0
proteus-mysql	36,484	54,257	108,610	133,051	133,868	134,560	134,560	Mcr	Jav	Utw	Non	Lin	My	Lin	Raw	Rea	0
pyramid	8,761	16,852	28,486	27,104	24,277	24,428	28,486	Ful	Py	Non	Mei	Lin	Pg	Lin	Ful	Rea	0
pyramid-py2	8,944	17,711	28,184	25,474	21,409	21,382	28,184	Ful	Py	Non	Mei	Lin	Pg	Lin	Ful	Rea	0
restexpress	23,518	30,109	40,243	33,109	25,564	25,074	40,243	Mcr	Jav	Nty	Non	Lin	Mo	Lin	Raw	Rea	0
restexpress-mysql-ra	167	167	83	165	83	660	660	Mcr	Jav	Nty	Non	Lin	My	Lin	Raw	Rea	121,507
revenj-jvm	45,466	114,123	169,879	172,092	149,804	124,703	172,092	Ful	Jav	Svt	Res	Lin	Pg	Lin	Ful	Rea	0
scruffy	20,022	18,844	22,867	13,013	14,343	14,308	22,867	Mcr	Sca	Utw	Non	Lin	Mo	Lin	Raw	Rea	54,214
silicon	31,543	52,922	68,051	68,945	66,189	60,939	68,945	Mcr	C++	Non	mic	Lin	My	Lin	Ful	Rea	0
spark	13,936	25,911	18,005	14,903	12,711	11,651	25,911	Mcr	Jav	Svt	Res	Lin	My	Lin	Ful	Rea	0
spring	7,884	15,130	13,669	12,878	12,764	12,975	15,130	Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	0
tapestry	12,394	21,136	28,444	32,316	31,902	31,462	32,316	Ful	Jav	Svt	Res	Lin	My	Lin	Ful	Rea	0
treefrog	19,150	35,993	54,144	59,000	56,738	50,952	59,000	Ful	C++	Non	Non	Lin	My	Lin	Mcr	Rea	0
treefrog-hybrid	10,316	18,620	28,186	38,538	49,538	52,684	52,684	Ful	C++	Non	Non	Lin	My	Lin	Mcr	Rea	2
treefrog-mongodb	21,217	29,704	48,132	65,980	70,902	73,047	73,047	Ful	C++	Non	Non	Lin	Mo	Lin	Mcr	Rea	0
treefrog-postgres	15,918	33,167	47,440	49,012	42,288	37,984	49,012	Ful	C++	Non	Non	Lin	Pg	Lin	Mcr	Rea	0
turbogears	6,577	12,557	18,908	16,813	15,027	14,720	18,908	Mcr	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
ulib-mongodb	37,786	107,107	157,603	175,101	173,700	174,404	175,101	Plt	C++	Non	ULi	Lin	Mo	Lin	Mcr	Rea	0
ulib-mysql	44,066	106,361	143,100	149,247	149,110	148,571	149,247	Plt	C++	Non	ULi	Lin	My	Lin	Mcr	Rea	0
ulib-postgres	46,595	104,798	174,473	176,064	182,112	180,491	182,112	Plt	C++	Non	ULi	Lin	Pg	Lin	Mcr	Rea	0
ulib-postgres_fit	40,501	119,447	166,642	180,821	187,519	185,711	187,519	Plt	C++	Non	ULi	Lin	Pg	Lin	Mcr	Rea	0
uvicorn	12,958	21,192	21,446	20,806	19,620	19,699	21,446	Plt	Py	Non	Non	Lin	Pg	Lin	Raw	Rea	0
vertx-web-mongodb	29,328	53,521	54,425	54,382	54,735	56,797	56,797	Mcr	Jav	vtx	Non	Lin	Mo	Lin	Raw	Rea	0
vertx-web-postgres	33,138	90,285	141,570	152,768	167,888	176,393	176,393	Mcr	Jav	vtx	Non	Lin	Pg	Lin	Raw	Rea	0
vertx-web-susom-post	26,283	62,643	96,592	112,497	113,360	112,976	113,360	Mcr	Jav	vtx	Non	Lin	Pg	Lin	Raw	Rea	0
web2py	3,203	5,924	8,276	8,587	8,030	8,320	8,587	Ful	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
web2py-optimized	4,042	7,353	11,435	11,374	10,404	10,513	11,435	Ful	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
webware	0	0	0	0	0	0	0	Mcr	Py	Non	Non	Lin	My	Lin	Ful	Rea	1,143
weppy	9,569	20,080	36,336	42,099	35,346	33,678	42,099	Ful	Py	Mei	Non	Lin	Pg	Lin	Ful	Rea	0
weppy-nginx-uwsgi	4,238	4,356	4,399	4,344	4,407	4,508	4,508	Ful	Py	uWS	ngx	Lin	Pg	Lin	Ful	Rea	0
weppy-py3	9,684	20,657	37,560	46,018	43,422	39,852	46,018	Ful	Py	Mei	Non	Lin	Pg	Lin	Ful	Rea	0
weppy-pypy2	1,642	2,363	3,345	3,496	2,758	3,030	3,496	Ful	Py	Tor	Non	Lin	Pg	Lin	Ful	Rea	0
wheezyweb	8,414	16,243	27,585	25,411	22,789	22,181	27,585	Mcr	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
wheezyweb-py3	— Did not complete —							Mcr	Py	Non	Mei	Lin	My	Lin	Ful	Rea	—
wicket	33,871	34,660	35,055	34,307	32,727	32,301	35,055	Ful	Jav	Svt	Res	Lin	My	Lin	Ful	Rea	0
wt	12,361	13,195	26,452	56,013	54,676	53,588	56,013	Ful	C++	Non	Non	Lin	My	Lin	Ful	Rea	0
wt-postgres	12,832	13,982	25,610	59,551	57,121	56,703	59,551	Ful	C++	Non	Non	Lin	Pg	Lin	Ful	Rea	0

Latency of database-access responses, Dell servers at ServerCentral

Framework	Average latency (lower is better)		σ (SD)	Max	Errors
ulib-postgres_fit	2.8 ms	0.1%	1.3 ms	53.9 ms	0
ulib-mongodb	3.0 ms	0.1%	1.4 ms	57.8 ms	0
light-4j-postgresql	3.0 ms	0.1%	1.7 ms	159.7 ms	0
ulib-postgres	3.0 ms	0.1%	2.1 ms	121.1 ms	0
proteus	3.0 ms	0.1%	1.6 ms	51.4 ms	0
cutelyst-thread-post	3.4 ms	0.1%	1.7 ms	32.3 ms	0
cutelyst-thread-post	3.4 ms	0.1%	1.7 ms	32.8 ms	0
vertx-web-postgres	3.4 ms	0.1%	4.5 ms	62.6 ms	0
ulib-mysql	3.5 ms	0.1%	1.3 ms	33.8 ms	0
vertx-postgres	3.6 ms	0.1%	6.0 ms	121.3 ms	0
cutelyst-pf-postgres	4.4 ms	0.1%	1.9 ms	26.0 ms	0
proteus-mysql	4.6 ms	0.1%	4.3 ms	96.6 ms	0
undertow-postgresql	4.7 ms	0.1%	9.5 ms	227.5 ms	0
go-prefork	4.7 ms	0.1%	10.9 ms	407.5 ms	0
cutelyst-thread-mysq	4.8 ms	0.1%	2.2 ms	39.6 ms	0
cutelyst-thread-mysq	4.9 ms	0.1%	2.4 ms	40.4 ms	0
vertx-web-susom-post	5.4 ms	0.1%	8.7 ms	220.1 ms	0
revenj-jvm	5.4 ms	0.1%	13.5 ms	463.1 ms	0
jooby	5.9 ms	0.1%	4.2 ms	60.6 ms	0
cutelyst-pf-mysql	6.2 ms	0.1%	2.8 ms	30.1 ms	0
cutelyst-nginx-postg	6.3 ms	0.1%	0.8 ms	42.5 ms	0
cutelyst-nginx-mysql	6.5 ms	0.1%	0.4 ms	30.2 ms	0
undertow-mysql	6.7 ms	0.1%	9.2 ms	188.5 ms	0
actframework-ebean-p	6.8 ms	0.1%	7.7 ms	212.3 ms	0
jawn	6.9 ms	0.1%	8.6 ms	128.1 ms	0
fasthttp-postgresql	7.0 ms	0.1%	6.8 ms	82.5 ms	0
actframework-hiberna	7.0 ms	0.1%	8.3 ms	212.9 ms	0
cpoll_cppsp-postgres	7.2 ms	0.1%	14.8 ms	458.2 ms	0
kami	7.4 ms	0.1%	7.6 ms	106.1 ms	0
actframework-morphia	7.7 ms	0.2%	8.7 ms	104.0 ms	0
actframework-ebean-m	7.8 ms	0.2%	9.2 ms	254.0 ms	0
actframework-hiberna	8.0 ms	0.2%	8.6 ms	226.0 ms	0
grizzly-jersey	8.2 ms	0.2%	10.8 ms	236.3 ms	0
scruffy	8.9 ms	0.2%	9.2 ms	158.4 ms	54,214
bottle-mysql-raw	9.2 ms	0.2%	13.4 ms	245.4 ms	0
cpoll_cppsp-raw	9.4 ms	0.2%	45.6 ms	1150.0 ms	0
go-interpolate	10.4 ms	0.2%	7.7 ms	88.0 ms	0
dropwizard-mysql	11.5 ms	0.2%	10.6 ms	232.1 ms	0
undertow-jersey-hika	11.5 ms	0.2%	11.1 ms	254.0 ms	0
cutelyst-pf-mysql-ep	11.9 ms	0.2%	12.3 ms	181.5 ms	0
morepath	11.9 ms	0.2%	13.8 ms	348.8 ms	0
dropwizard-postgres	12.1 ms	0.2%	10.5 ms	174.5 ms	0
revel-raw	12.1 ms	0.2%	11.1 ms	113.9 ms	0
gin	12.4 ms	0.2%	12.9 ms	282.1 ms	0
falcore	12.6 ms	0.2%	11.0 ms	133.6 ms	0
beego-orm-mysql	12.6 ms	0.2%	12.5 ms	252.3 ms	0
cpoll_cppsp-postgres	12.8 ms	0.2%	71.2 ms	1940.0 ms	0
undertow-mongodb-asy	12.8 ms	0.3%	13.6 ms	272.5 ms	0
revel-qbs	12.9 ms	0.3%	11.5 ms	125.2 ms	0
tornado-pypy2	12.9 ms	0.3%	17.4 ms	500.7 ms	0
servlet-raw	13.0 ms	0.3%	18.6 ms	269.8 ms	0
revel-jet	13.0 ms	0.3%	11.4 ms	115.4 ms	0
api_hour-mysql	13.0 ms	0.3%	21.6 ms	822.1 ms	0
wt	13.1 ms	0.3%	39.5 ms	880.3 ms	0
weppy-py3	13.3 ms	0.3%	13.3 ms	234.3 ms	0
wt-postgres	13.4 ms	0.3%	41.5 ms	765.1 ms	0
go-postgres	13.9 ms	0.3%	10.0 ms	95.0 ms	0
play2-scala-anorm-li	13.9 ms	0.3%	21.9 ms	337.1 ms	0
treefrog-hybrid	14.1 ms	0.3%	43.9 ms	918.2 ms	2
treefrog-mongodb	14.1 ms	0.3%	60.4 ms	1640.0 ms	0
echo	14.3 ms	0.3%	10.3 ms	99.4 ms	0
go	14.5 ms	0.3%	10.1 ms	93.0 ms	0
vertx-web-mongodb	15.0 ms	0.3%	17.2 ms	247.5 ms	0
aiohttp	15.6 ms	0.3%	17.9 ms	428.2 ms	0
aiohttp-pg-raw	15.6 ms	0.3%	13.5 ms	597.5 ms	0
fintrospect	16.0 ms	0.3%	92.3 ms	1630.0 ms	0
flask-mysql-raw	16.2 ms	0.3%	17.3 ms	387.0 ms	0
play2-scala-anorm-li	16.3 ms	0.3%	30.7 ms	833.5 ms	0
fasthttp	16.3 ms	0.3%	19.9 ms	430.3 ms	0
aah	16.6 ms	0.3%	12.9 ms	236.7 ms	0
play2-java-jpa-hikar	16.6 ms	0.3%	22.2 ms	445.3 ms	0
activeweb	16.7 ms	0.3%	25.4 ms	689.6 ms	0
weppy	16.9 ms	0.3%	16.7 ms	414.3 ms	0
activeweb-jackson	17.1 ms	0.3%	24.4 ms	529.5 ms	0
dropwizard-mongodb	17.2 ms	0.3%	13.8 ms	234.1 ms	0
play2-java-jpa-hikar	17.8 ms	0.3%	22.4 ms	437.6 ms	0
treefrog	18.0 ms	0.4%	63.6 ms	1610.0 ms	0
http4s	18.1 ms	0.4%	88.8 ms	1640.0 ms	0
servlet-postgres-raw	18.5 ms	0.4%	34.6 ms	412.6 ms	0
goji	18.8 ms	0.4%	15.8 ms	148.9 ms	0
wicket	18.9 ms	0.4%	20.8 ms	478.9 ms	0
wildfly-ee7	19.4 ms	0.4%	30.4 ms	345.3 ms	0
akka-http	19.9 ms	0.4%	68.1 ms	1010.0 ms	0
undertow-jersey-c3p0	20.5 ms	0.4%	9.1 ms	225.7 ms	0
api_hour	20.8 ms	0.4%	28.2 ms	458.5 ms	0
restexpress	20.9 ms	0.4%	29.6 ms	882.2 ms	0
treefrog-postgres	21.1 ms	0.4%	75.3 ms	1690.0 ms	0
django-py3	21.5 ms	0.4%	20.2 ms	301.2 ms	0
tapestry	21.7 ms	0.4%	33.3 ms	1020.0 ms	0
minijax	22.7 ms	0.4%	92.8 ms	1650.0 ms	0
pyramid	23.4 ms	0.5%	22.2 ms	397.9 ms	0
play1	24.7 ms	0.5%	102.7 ms	1610.0 ms	0
wheezyweb	25.0 ms	0.5%	22.5 ms	430.3 ms	0
play2-java-ebean-hik	25.3 ms	0.5%	16.9 ms	248.0 ms	0
dropwizard-jdbi-post	26.1 ms	0.5%	65.4 ms	595.6 ms	0
blade	26.3 ms	0.5%	21.4 ms	256.7 ms	0
pyramid-py2	26.5 ms	0.5%	24.9 ms	474.4 ms	0
play2-java-ebean-hik	28.6 ms	0.6%	40.9 ms	872.1 ms	0
django-postgresql	29.2 ms	0.6%	25.9 ms	619.5 ms	0
bottle	30.4 ms	0.6%	25.0 ms	470.5 ms	0
flask-py3	30.4 ms	0.6%	26.5 ms	458.9 ms	0
uvicorn	30.6 ms	0.6%	33.8 ms	587.4 ms	0
bottle-py3	30.7 ms	0.6%	27.6 ms	694.4 ms	0
tornado-py3	31.3 ms	0.6%	16.2 ms	405.0 ms	0
django	31.7 ms	0.6%	29.7 ms	575.7 ms	0
ffead-cpp-nginx-mong	32.5 ms	0.6%	3.5 ms	267.8 ms	0
tornado	33.1 ms	0.6%	20.1 ms	584.2 ms	0
ffead-cpp-apache-mon	33.1 ms	0.6%	2.5 ms	63.3 ms	0
go-mongo	33.5 ms	0.7%	3.9 ms	86.9 ms	0
ffead-cpp	33.6 ms	0.7%	3.5 ms	270.9 ms	0
flask	34.2 ms	0.7%	28.0 ms	586.2 ms	0
cppcms	35.4 ms	0.7%	3.5 ms	43.8 ms	0
cppcms-postgres	35.4 ms	0.7%	4.5 ms	45.3 ms	0
cherrypy-py3	35.4 ms	0.7%	6.8 ms	278.1 ms	0
turbogears	37.4 ms	0.7%	32.8 ms	462.3 ms	0
silicon	38.3 ms	0.7%	228.2 ms	3380.0 ms	0
spring	40.3 ms	0.8%	21.1 ms	198.6 ms	0
cppcms-direct	46.6 ms	0.9%	5.6 ms	249.4 ms	0
undertow-mongodb	48.8 ms	1.0%	115.7 ms	787.5 ms	0
spark	48.8 ms	1.0%	44.6 ms	684.4 ms	0
gemini-mysql	49.6 ms	1.0%	99.1 ms	1090.0 ms	0
web2py-optimized	50.8 ms	1.0%	38.5 ms	701.1 ms	0
cherrypy	51.0 ms	1.0%	11.3 ms	271.8 ms	319
ninja-standalone	57.7 ms	1.1%	32.5 ms	878.0 ms	0
play2-java-jooq-hika	60.9 ms	1.2%	167.2 ms	1120.0 ms	0
tornado-postgresql-r	62.0 ms	1.2%	13.7 ms	268.6 ms	0
web2py	66.6 ms	1.3%	55.4 ms	681.0 ms	0
ffead-cpp-postgresql	75.1 ms	1.5%	11.1 ms	291.2 ms	0
ffead-cpp-nginx-post	76.9 ms	1.5%	4.3 ms	289.1 ms	0
ffead-cpp-apache-pos	77.5 ms	1.5%	9.0 ms	295.4 ms	0
play2-java-jooq-hika	96.8 ms	1.9%	277.2 ms	1710.0 ms	0
play2-scala-slick-ne	108.9 ms	2.1%	147.8 ms	1100.0 ms	0
weppy-nginx-uwsgi	113.5 ms	2.2%	13.4 ms	344.5 ms	0
bottle-nginx-uwsgi	119.4 ms	2.3%	17.9 ms	342.8 ms	0
flask-nginx-uwsgi	121.8 ms	2.4%	16.1 ms	356.7 ms	0
gemini-postgres	156.4 ms	3.1%	299.2 ms	1650.0 ms	0
play2-scala-slick	175.2 ms	3.4%	556.5 ms	6680.0 ms	0
weppy-pypy2	197.0 ms	3.9%	197.6 ms	2630.0 ms	0
flask-pypy2-mysql-ra	231.4 ms	4.5%	174.9 ms	1930.0 ms	0
restexpress-mysql-ra	278.1 ms	5.4%	607.2 ms	3340.0 ms	121,507
play2-scala-reactive	334.0 ms	6.5%	331.7 ms	1300.0 ms	0
play2-scala-reactive	393.2 ms	7.7%	443.6 ms	1880.0 ms	0
bottle-pypy2	413.4 ms	8.1%	295.1 ms	3230.0 ms	0
flask-pypy2	519.2 ms	10.2%	317.6 ms	2470.0 ms	0
klein	1390.0 ms	27.2%	693.6 ms	5800.0 ms	0
webware	5110.0 ms	100.0%	1520.0 ms	7000.0 ms	1,143

Framework overhead of database-access responses, Dell servers at ServerCentral

Framework best		vs	Platform best		Ratio (higher is better)
tornado-pypy2	47,638	vs	tornado	17,015	280.0%
fasthttp-postgresql	137,073	vs	go	58,969	232.4%
go-prefork	124,834	vs	go	58,969	211.7%
undertow-jersey-hika	55,937	vs	undertow-jersey-c3p0	31,657	176.7%
kami	101,111	vs	go	58,969	171.5%
go-interpolate	98,849	vs	go	58,969	167.6%
treefrog-mongodb	73,047	vs	treefrog	59,000	123.8%
fasthttp	66,424	vs	go	58,969	112.6%
wt-postgres	59,551	vs	wt	56,013	106.3%
falcore	60,414	vs	go	58,969	102.5%
goji	59,755	vs	go	58,969	101.3%
beego-orm-mysql	59,302	vs	go	58,969	100.6%
go-postgres	59,208	vs	go	58,969	100.4%
gin	58,883	vs	go	58,969	99.9%
dropwizard-postgres	55,163	vs	servlet-postgres-raw	57,211	96.4%
tornado-py3	16,362	vs	tornado	17,015	96.2%
revel-raw	55,320	vs	go	58,969	93.8%
echo	54,578	vs	go	58,969	92.6%
revel-jet	53,746	vs	go	58,969	91.1%
revel-qbs	53,019	vs	go	58,969	89.9%
servlet-postgres-raw	57,211	vs	servlet-raw	63,712	89.8%
treefrog-hybrid	52,684	vs	treefrog	59,000	89.3%
dropwizard-mysql	56,521	vs	servlet-raw	63,712	88.7%
dropwizard-jdbi-post	36,582	vs	servlet-postgres-raw	57,211	63.9%
undertow-jersey-c3p0	31,657	vs	undertow-jersey-hika	55,937	56.6%
aah	31,799	vs	go	58,969	53.9%
tornado-postgresql-r	8,176	vs	tornado	17,015	48.1%
play2-scala-anorm-li	38,457	vs	akka-http	87,715	43.8%
play2-java-jpa-hikar	34,686	vs	akka-http	87,715	39.5%
play2-java-jooq-hika	24,532	vs	akka-http	87,715	28.0%
go-mongo	15,802	vs	go	58,969	26.8%
play2-java-ebean-hik	20,368	vs	akka-http	87,715	23.2%
play2-scala-slick	14,990	vs	akka-http	87,715	17.1%
play2-scala-reactive	1,894	vs	akka-http	87,715	2.2%

Best (bar chart)

Data table

Latency

Framework overhead

Best (bar chart)

Data table

Latency

Framework overhead

Requirements summary

In this test, each request is processed by fetching a single row from a simple database table. That row is then serialized as a JSON response.

Example response:

HTTP/1.1 200 OK Content-Length: 32 Content-Type: application/json Server: Example Date: Wed, 17 Apr 2013 12:00:00 GMT {"id":3217,"randomNumber":2149}

For a more detailed description of the requirements, see the Source Code and Requirements section.

Responses per second at 20 queries per request, Dell servers at ServerCentral(149 tests)

Framework	Performance (higher is better)		Cls	Lng	Plt	FE	Aos	DB	Dos	Orm	IA	Errors
undertow-postgresql	9,514	100.0%(92.0%)	Plt	Jav	Utw	Non	Lin	Pg	Lin	Raw	Rea	0
wt-postgres	9,349	98.3%(90.4%)	Ful	C++	Non	Non	Lin	Pg	Lin	Ful	Rea	0
jooby	9,196	96.7%(88.9%)	Ful	Jav	Nty	Non	Lin	Pg	Lin	Raw	Rea	0
ulib-postgres_fit	9,171	96.4%(88.7%)	Plt	C++	Non	ULi	Lin	Pg	Lin	Mcr	Rea	0
vertx-web-susom-post	9,130	96.0%(88.3%)	Mcr	Jav	vtx	Non	Lin	Pg	Lin	Raw	Rea	0
vertx-web-postgres	9,012	94.7%(87.2%)	Mcr	Jav	vtx	Non	Lin	Pg	Lin	Raw	Rea	0
light-4j-postgresql	8,855	93.1%(85.6%)	Plt	Jav	Lig	Non	Lin	Pg	Lin	Raw	Rea	0
ulib-mongodb	8,745	91.9%(84.6%)	Plt	C++	Non	ULi	Lin	Mo	Lin	Mcr	Rea	0
ulib-postgres	8,699	91.4%(84.1%)	Plt	C++	Non	ULi	Lin	Pg	Lin	Mcr	Rea	0
vertx-postgres	8,697	91.4%(84.1%)	Plt	Jav	Ver	Non	Lin	Pg	Lin	Raw	Rea	0
jawn	8,332	87.6%(80.6%)	Ful	Jav	Svt	Utw	Lin	Pg	Lin	Raw	Rea	0
cutelyst-thread-post	7,808	82.1%(75.5%)	Ful	C++	Qt	Non	Lin	Pg	Lin	Raw	Rea	0
cutelyst-thread-post	7,738	81.3%(74.8%)	Ful	C++	Qt	Non	Lin	Pg	Lin	Raw	Rea	0
cutelyst-nginx-postg	7,630	80.2%(73.8%)	Ful	C++	Qt	ngx	Lin	Pg	Lin	Raw	Rea	0
cutelyst-pf-postgres	7,563	79.5%(73.1%)	Ful	C++	Qt	Non	Lin	Pg	Lin	Raw	Rea	0
undertow-mysql	7,286	76.6%(70.5%)	Plt	Jav	Utw	Non	Lin	My	Lin	Raw	Rea	0
ulib-mysql	7,187	75.5%(69.5%)	Plt	C++	Non	ULi	Lin	My	Lin	Mcr	Rea	0
cpoll_cppsp-postgres	7,177	75.4%(69.4%)	Plt	C++	Non	Non	Lin	Pg	Lin	Raw	Rea	0
treefrog-mongodb	6,688	70.3%(64.7%)	Ful	C++	Non	Non	Lin	Mo	Lin	Mcr	Rea	0
servlet-postgres-raw	6,341	66.6%(61.3%)	Plt	Jav	Svt	Res	Lin	Pg	Lin	Raw	Rea	0
cpoll_cppsp-raw	6,333	66.6%(61.2%)	Plt	C++	Non	Non	Lin	My	Lin	Raw	Rea	0
cpoll_cppsp-postgres	6,197	65.1%(59.9%)	Plt	C++	Non	Non	Lin	Pg	Lin	Raw	Rea	0
revenj-jvm	6,133	64.5%(59.3%)	Ful	Jav	Svt	Res	Lin	Pg	Lin	Ful	Rea	0
gemini-postgres	6,109	64.2%(59.1%)	Ful	Jav	Svt	Res	Lin	Pg	Lin	Mcr	Rea	0
fasthttp-postgresql	6,091	64.0%(58.9%)	Plt	Go	Non	Non	Lin	Pg	Lin	Raw	Rea	0
go-prefork	6,072	63.8%(58.7%)	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
silicon	6,004	63.1%(58.1%)	Mcr	C++	Non	mic	Lin	My	Lin	Ful	Rea	53
akka-http	5,897	62.0%(57.0%)	Mcr	Sca	Akk	Non	Lin	My	Lin	Raw	Rea	0
bottle-mysql-raw	5,737	60.3%(55.5%)	Mcr	Py	Mei	Non	Lin	My	Lin	Raw	Rea	0
flask-mysql-raw	5,555	58.4%(53.7%)	Mcr	Py	Mei	Non	Lin	My	Lin	Raw	Rea	0
aiohttp-pg-raw	5,443	57.2%(52.6%)	Mcr	Py	asy	Gun	Lin	Pg	Lin	Raw	Rea	0
wt	5,433	57.1%(52.5%)	Ful	C++	Non	Non	Lin	My	Lin	Ful	Rea	0
cutelyst-thread-mysq	5,368	56.4%(51.9%)	Ful	C++	Qt	Non	Lin	My	Lin	Raw	Rea	0
http4s	5,366	56.4%(51.9%)	Mcr	Sca	NIO	bla	Lin	Pg	Lin	Raw	Rea	0
cutelyst-thread-mysq	5,336	56.1%(51.6%)	Ful	C++	Qt	Non	Lin	My	Lin	Raw	Rea	0
wildfly-ee7	5,292	55.6%(51.2%)	Ful	Jav	Svt	Wil	Lin	My	Lin	Ful	Rea	0
api_hour-mysql	5,189	54.5%(50.2%)	Mcr	Py	asy	Gun	Lin	My	Lin	Raw	Rea	0
fintrospect	5,168	54.3%(50.0%)	Mcr	Sca	Nty	Non	Lin	My	Lin	Raw	Rea	0
servlet-raw	5,100	53.6%(49.3%)	Plt	Jav	Svt	Res	Lin	My	Lin	Raw	Rea	0
kami	4,938	51.9%(47.8%)	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
actframework-hiberna	4,812	50.6%(46.5%)	Ful	Jav	Utw	Non	Lin	Pg	Lin	Ful	Rea	0
cppcms	4,739	49.8%(45.8%)	Plt	C++	Non	Non	Lin	My	Lin	Raw	Rea	0
web2py-optimized	4,712	49.5%(45.6%)	Ful	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
cppcms-direct	4,707	49.5%(45.5%)	Plt	C++	Non	Non	Lin	My	Lin	Raw	Rea	0
cutelyst-nginx-mysql	4,696	49.4%(45.4%)	Ful	C++	Qt	ngx	Lin	My	Lin	Raw	Rea	0
actframework-morphia	4,672	49.1%(45.2%)	Ful	Jav	Utw	Non	Lin	Mo	Lin	Ful	Rea	0
cutelyst-pf-mysql	4,659	49.0%(45.1%)	Ful	C++	Qt	Non	Lin	My	Lin	Raw	Rea	0
play1	4,640	48.8%(44.9%)	Ful	Jav	Nty	Non	Lin	My	Lin	Ful	Rea	0
actframework-ebean-p	4,609	48.4%(44.6%)	Ful	Jav	Utw	Non	Lin	Pg	Lin	Ful	Rea	0
cutelyst-pf-mysql-ep	4,601	48.4%(44.5%)	Ful	C++	Qt	Non	Lin	My	Lin	Raw	Rea	0
morepath	4,247	44.6%(41.1%)	Mcr	Py	Mei	Gun	Lin	Pg	Lin	Ful	Rea	0
actframework-hiberna	4,236	44.5%(41.0%)	Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	0
actframework-ebean-m	4,046	42.5%(39.1%)	Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	0
grizzly-jersey	3,878	40.8%(37.5%)	Mcr	Jav	JAX	Non	Lin	My	Lin	Ful	Rea	0
weppy-py3	3,843	40.4%(37.2%)	Ful	Py	Mei	Non	Lin	Pg	Lin	Ful	Rea	0
undertow-jersey-hika	3,652	38.4%(35.3%)	Plt	Jav	JAX	Non	Lin	My	Lin	Ful	Rea	0
ninja-standalone	3,629	38.1%(35.1%)	Ful	Jav	Jty	Non	Lin	My	Lin	Ful	Rea	0
go-interpolate	3,467	36.4%(33.5%)	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
api_hour	3,458	36.3%(33.4%)	Mcr	Py	asy	Gun	Lin	Pg	Lin	Raw	Rea	0
tornado-pypy2	3,438	36.1%(33.2%)	Plt	Py	Non	Tor	Lin	Mo	Lin	Raw	Rea	0
tapestry	3,204	33.7%(31.0%)	Ful	Jav	Svt	Res	Lin	My	Lin	Ful	Rea	0
weppy-nginx-uwsgi	3,173	33.4%(30.7%)	Ful	Py	uWS	ngx	Lin	Pg	Lin	Ful	Rea	0
minijax	3,135	33.0%(30.3%)	Ful	Jav	JAX	Jty	Lin	My	Lin	Ful	Rea	0
dropwizard-postgres	3,092	32.5%(29.9%)	Ful	Jav	JAX	Jty	Lin	Pg	Lin	Ful	Rea	0
weppy	3,029	31.8%(29.3%)	Ful	Py	Mei	Non	Lin	Pg	Lin	Ful	Rea	0
dropwizard-mysql	2,970	31.2%(28.7%)	Ful	Jav	JAX	Jty	Lin	My	Lin	Ful	Rea	0
gemini-mysql	2,910	30.6%(28.1%)	Ful	Jav	Svt	Res	Lin	My	Lin	Mcr	Rea	0
undertow-mongodb-asy	2,904	30.5%(28.1%)	Plt	Jav	Utw	Non	Lin	Mo	Lin	Raw	Rea	266
gin	2,870	30.2%(27.8%)	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
goji	2,850	30.0%(27.6%)	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
vertx-web-mongodb	2,845	29.9%(27.5%)	Mcr	Jav	vtx	Non	Lin	Mo	Lin	Raw	Rea	12
play2-scala-anorm-li	2,826	29.7%(27.3%)	Ful	Sca	Akk	Non	Lin	My	Lin	Ful	Rea	0
falcore	2,797	29.4%(27.1%)	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
play2-scala-anorm-li	2,796	29.4%(27.0%)	Ful	Sca	Nty	Non	Lin	My	Lin	Ful	Rea	0
cppcms-postgres	2,782	29.2%(26.9%)	Plt	C++	Non	Non	Lin	Pg	Lin	Raw	Rea	0
revel-raw	2,716	28.5%(26.3%)	Ful	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
web2py	2,699	28.4%(26.1%)	Ful	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
revel-qbs	2,603	27.4%(25.2%)	Ful	Go	Non	Non	Lin	My	Lin	Mcr	Rea	0
revel-jet	2,580	27.1%(25.0%)	Ful	Go	Non	Non	Lin	My	Lin	Mcr	Rea	0
wicket	2,559	26.9%(24.7%)	Ful	Jav	Svt	Res	Lin	My	Lin	Ful	Rea	0
treefrog-hybrid	2,497	26.2%(24.1%)	Ful	C++	Non	Non	Lin	My	Lin	Mcr	Rea	2
flask-py3	2,485	26.1%(24.0%)	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
undertow-mongodb	2,483	26.1%(24.0%)	Plt	Jav	Utw	Non	Lin	Mo	Lin	Raw	Rea	10,091
activeweb-jackson	2,443	25.7%(23.6%)	Ful	Jav	Svt	Non	Lin	My	Lin	Mcr	Rea	0
aiohttp	2,372	24.9%(22.9%)	Mcr	Py	asy	Gun	Lin	Pg	Lin	Ful	Rea	0
fasthttp	2,350	24.7%(22.7%)	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
play2-java-jpa-hikar	2,311	24.3%(22.4%)	Ful	Jav	Akk	Non	Lin	My	Lin	Ful	Rea	0
go	2,279	24.0%(22.0%)	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
play2-java-jpa-hikar	2,271	23.9%(22.0%)	Ful	Jav	Nty	Non	Lin	My	Lin	Ful	Rea	0
go-postgres	2,265	23.8%(21.9%)	Plt	Go	Non	Non	Lin	Pg	Lin	Raw	Rea	0
flask-nginx-uwsgi	2,247	23.6%(21.7%)	Mcr	Py	Non	ngx	Lin	My	Lin	Ful	Rea	0
echo	2,247	23.6%(21.7%)	Mcr	Go	Non	Non	Lin	Pg	Lin	Raw	Rea	0
activeweb	2,245	23.6%(21.7%)	Ful	Jav	Svt	Non	Lin	My	Lin	Mcr	Rea	0
aah	2,221	23.3%(21.5%)	Ful	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
treefrog	2,216	23.3%(21.4%)	Ful	C++	Non	Non	Lin	My	Lin	Mcr	Rea	0
dropwizard-jdbi-post	2,108	22.2%(20.4%)	Ful	Jav	JAX	Jty	Lin	Pg	Lin	Mcr	Rea	102
flask	2,074	21.8%(20.1%)	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
django-py3	2,037	21.4%(19.7%)	Ful	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
play2-java-jooq-hika	2,028	21.3%(19.6%)	Ful	Jav	Nty	Non	Lin	My	Lin	Ful	Rea	0
uvicorn	1,985	20.9%(19.2%)	Plt	Py	Non	Non	Lin	Pg	Lin	Raw	Rea	0
bottle-nginx-uwsgi	1,839	19.3%(17.8%)	Mcr	Py	uWS	ngx	Lin	My	Lin	Ful	Rea	0
treefrog-postgres	1,830	19.2%(17.7%)	Ful	C++	Non	Non	Lin	Pg	Lin	Mcr	Rea	0
bottle-py3	1,750	18.4%(16.9%)	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
dropwizard-mongodb	1,741	18.3%(16.8%)	Ful	Jav	JAX	Jty	Lin	Mo	Lin	Ful	Rea	15,783
wheezyweb	1,734	18.2%(16.8%)	Mcr	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
pyramid	1,733	18.2%(16.8%)	Ful	Py	Non	Mei	Lin	Pg	Lin	Ful	Rea	0
django-postgresql	1,714	18.0%(16.6%)	Ful	Py	Non	Mei	Lin	Pg	Lin	Ful	Rea	0
bottle	1,623	17.1%(15.7%)	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
django	1,605	16.9%(15.5%)	Ful	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
turbogears	1,593	16.7%(15.4%)	Mcr	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
tornado-py3	1,589	16.7%(15.4%)	Plt	Py	Non	Tor	Lin	Mo	Lin	Raw	Rea	0
weppy-pypy2	1,585	16.7%(15.3%)	Ful	Py	Tor	Non	Lin	Pg	Lin	Ful	Rea	0
restexpress	1,526	16.0%(14.8%)	Mcr	Jav	Nty	Non	Lin	Mo	Lin	Raw	Rea	0
spring	1,516	15.9%(14.7%)	Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	0
flask-pypy2-mysql-ra	1,506	15.8%(14.6%)	Mcr	Py	Tor	Non	Lin	My	Lin	Raw	Rea	0
tornado	1,501	15.8%(14.5%)	Plt	Py	Non	Tor	Lin	Mo	Lin	Raw	Rea	0
pyramid-py2	1,449	15.2%(14.0%)	Ful	Py	Non	Mei	Lin	Pg	Lin	Ful	Rea	0
play2-java-ebean-hik	1,272	13.4%(12.3%)	Ful	Jav	Akk	Non	Lin	My	Lin	Ful	Rea	0
play2-java-ebean-hik	1,258	13.2%(12.2%)	Ful	Jav	Nty	Non	Lin	My	Lin	Ful	Rea	0
play2-scala-reactive	1,250	13.1%(12.1%)	Ful	Sca	Nty	Non	Lin	Mo	Lin	Ful	Rea	0
play2-java-jooq-hika	1,241	13.0%(12.0%)	Ful	Jav	Akk	Non	Lin	My	Lin	Ful	Rea	0
blade	1,111	11.7%(10.7%)	Ful	Jav	Non	Non	Lin	My	Lin	Ful	Rea	0
beego-orm-mysql	1,087	11.4%(10.5%)	Ful	Go	Non	Non	Lin	My	Lin	Mcr	Rea	0
play2-scala-reactive	967	10.2%(9.4%)	Ful	Sca	Akk	Non	Lin	Mo	Lin	Ful	Rea	0
go-mongo	785	8.3%(7.6%)	Plt	Go	Non	Non	Lin	Mo	Lin	Raw	Rea	0
play2-scala-slick	689	7.2%(6.7%)	Ful	Sca	Akk	Non	Lin	My	Lin	Ful	Rea	247
play2-scala-slick-ne	651	6.8%(6.3%)	Ful	Sca	Nty	Non	Lin	My	Lin	Ful	Rea	0
restexpress-mysql-ra	516	5.4%(5.0%)	Mcr	Jav	Nty	Non	Lin	My	Lin	Raw	Rea	76,363
undertow-jersey-c3p0	454	4.8%(4.4%)	Plt	Jav	JAX	Non	Lin	My	Lin	Ful	Rea	0
spark	384	4.0%(3.7%)	Mcr	Jav	Svt	Res	Lin	My	Lin	Ful	Rea	0
tornado-postgresql-r	376	4.0%(3.6%)	Plt	Py	Non	Tor	Lin	Pg	Lin	Raw	Rea	911
ffead-cpp-apache-pos	363	3.8%(3.5%)	Ful	C++	ffe	Apc	Lin	Pg	Lin	Ful	Rea	0
ffead-cpp-nginx-post	362	3.8%(3.5%)	Ful	C++	ffe	ngx	Lin	Pg	Lin	Ful	Rea	0
bottle-pypy2	283	3.0%(2.7%)	Mcr	Py	Tor	Non	Lin	My	Lin	Ful	Rea	0
flask-pypy2	274	2.9%(2.6%)	Mcr	Py	Non	Tor	Lin	My	Lin	Ful	Rea	0
cherrypy-py3	31	0.3%(0.3%)	Mcr	Py	Non	Non	Lin	My	Lin	Ful	Rea	366
cherrypy	25	0.3%(0.2%)	Mcr	Py	Non	Non	Lin	My	Lin	Ful	Rea	218
klein	20	0.2%(0.2%)	Mcr	Py	Non	Twi	Lin	My	Lin	Ful	Rea	443
ffead-cpp	0	0.0%(0.0%)	Ful	C++	ffe	ffe	Lin	Mo	Lin	Ful	Rea	0
ffead-cpp-apache-mon	0	0.0%(0.0%)	Ful	C++	ffe	Apc	Lin	Mo	Lin	Ful	Rea	0
ffead-cpp-apache-mys	0	0.0%(0.0%)	Ful	C++	ffe	Apc	Lin	My	Lin	Ful	Rea	0
ffead-cpp-mysql	0	0.0%(0.0%)	Ful	C++	ffe	ffe	Lin	My	Lin	Ful	Rea	0
ffead-cpp-nginx-mong	0	0.0%(0.0%)	Ful	C++	ffe	ngx	Lin	Mo	Lin	Ful	Rea	0
ffead-cpp-nginx-mysq	0	0.0%(0.0%)	Ful	C++	ffe	ngx	Lin	My	Lin	Ful	Rea	0
ffead-cpp-postgresql	0	0.0%(0.0%)	Ful	C++	ffe	ffe	Lin	Pg	Lin	Ful	Rea	0
webware	0	0.0%(0.0%)	Mcr	Py	Non	Non	Lin	My	Lin	Ful	Rea	807
actframework-eclipse	—	Did not complete	Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	—
actframework-eclipse	—	Did not complete	Ful	Jav	Utw	Non	Lin	Pg	Lin	Ful	Rea	—
wheezyweb-py3	—	Did not complete	Mcr	Py	Non	Mei	Lin	My	Lin	Ful	Rea	—

Responses per second, multiple queries, Dell servers at ServerCentral

Framework	1	5	10	15	20	Cls	Lng	Plt	FE	Aos	DB	Dos	Orm	IA	Errors
aah	31,892	8,395	4,145	2,928	2,221	Ful	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
actframework-ebean-m	79,047	16,291	8,095	5,302	4,046	Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	0
actframework-ebean-p	91,516	18,445	9,270	6,210	4,609	Ful	Jav	Utw	Non	Lin	Pg	Lin	Ful	Rea	0
actframework-hiberna	68,960	16,489	8,539	5,697	4,236	Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	0
actframework-hiberna	90,455	18,998	9,483	6,386	4,812	Ful	Jav	Utw	Non	Lin	Pg	Lin	Ful	Rea	0
actframework-morphia	88,461	18,003	9,116	6,138	4,672	Ful	Jav	Utw	Non	Lin	Mo	Lin	Ful	Rea	0
actframework-eclipse	— Did not complete —						Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	—
actframework-eclipse	— Did not complete —						Ful	Jav	Utw	Non	Lin	Pg	Lin	Ful	Rea	—
activeweb	14,886	7,605	4,282	2,967	2,245	Ful	Jav	Svt	Non	Lin	My	Lin	Mcr	Rea	0
activeweb-jackson	46,645	9,750	4,902	3,268	2,443	Ful	Jav	Svt	Non	Lin	My	Lin	Mcr	Rea	0
aiohttp	40,210	8,877	4,800	3,093	2,372	Mcr	Py	asy	Gun	Lin	Pg	Lin	Ful	Rea	0
aiohttp-pg-raw	33,172	15,964	9,676	6,956	5,443	Mcr	Py	asy	Gun	Lin	Pg	Lin	Raw	Rea	0
akka-http	82,718	21,660	11,196	7,741	5,897	Mcr	Sca	Akk	Non	Lin	My	Lin	Raw	Rea	0
api_hour	35,036	12,300	6,137	4,415	3,458	Mcr	Py	asy	Gun	Lin	Pg	Lin	Raw	Rea	0
api_hour-mysql	42,650	16,643	9,396	6,586	5,189	Mcr	Py	asy	Gun	Lin	My	Lin	Raw	Rea	0
beego-orm-mysql	16,686	4,047	2,148	1,492	1,087	Ful	Go	Non	Non	Lin	My	Lin	Mcr	Rea	0
blade	21,526	4,364	2,179	1,468	1,111	Ful	Jav	Non	Non	Lin	My	Lin	Ful	Rea	0
bottle	16,049	5,495	3,100	2,160	1,623	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
bottle-mysql-raw	64,861	21,378	11,358	7,649	5,737	Mcr	Py	Mei	Non	Lin	My	Lin	Raw	Rea	0
bottle-nginx-uwsgi	4,880	5,688	3,340	2,367	1,839	Mcr	Py	uWS	ngx	Lin	My	Lin	Ful	Rea	0
bottle-py3	17,569	5,925	3,361	2,325	1,750	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
bottle-pypy2	2,007	632	376	322	283	Mcr	Py	Tor	Non	Lin	My	Lin	Ful	Rea	0
cpoll_cppsp-postgres	95,015	25,479	13,515	9,315	7,177	Plt	C++	Non	Non	Lin	Pg	Lin	Raw	Rea	0
cpoll_cppsp-postgres	127,805	25,205	12,472	8,259	6,197	Plt	C++	Non	Non	Lin	Pg	Lin	Raw	Rea	0
cpoll_cppsp-raw	130,474	25,689	12,799	8,483	6,333	Plt	C++	Non	Non	Lin	My	Lin	Raw	Rea	0
cutelyst-nginx-mysql	78,640	18,169	9,326	6,245	4,696	Ful	C++	Qt	ngx	Lin	My	Lin	Raw	Rea	0
cutelyst-nginx-postg	82,596	29,213	14,957	10,089	7,630	Ful	C++	Qt	ngx	Lin	Pg	Lin	Raw	Rea	0
cutelyst-pf-mysql	82,880	18,372	9,293	6,204	4,659	Ful	C++	Qt	Non	Lin	My	Lin	Raw	Rea	0
cutelyst-pf-mysql-ep	53,527	17,554	9,087	6,119	4,601	Ful	C++	Qt	Non	Lin	My	Lin	Raw	Rea	0
cutelyst-pf-postgres	113,879	29,580	15,019	10,082	7,563	Ful	C++	Qt	Non	Lin	Pg	Lin	Raw	Rea	0
cutelyst-thread-mysq	106,772	21,484	10,743	7,160	5,368	Ful	C++	Qt	Non	Lin	My	Lin	Raw	Rea	0
cutelyst-thread-mysq	106,011	21,505	10,733	7,153	5,336	Ful	C++	Qt	Non	Lin	My	Lin	Raw	Rea	0
cutelyst-thread-post	154,906	31,445	15,650	10,383	7,738	Ful	C++	Qt	Non	Lin	Pg	Lin	Raw	Rea	0
cutelyst-thread-post	156,047	31,540	15,612	10,430	7,808	Ful	C++	Qt	Non	Lin	Pg	Lin	Raw	Rea	0
django	16,888	5,490	3,006	2,041	1,605	Ful	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
django-postgresql	17,181	5,989	3,164	2,256	1,714	Ful	Py	Non	Mei	Lin	Pg	Lin	Ful	Rea	0
django-py3	24,337	7,285	3,879	2,672	2,037	Ful	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
dropwizard-jdbi-post	42,641	8,934	4,678	2,853	2,108	Ful	Jav	JAX	Jty	Lin	Pg	Lin	Mcr	Rea	102
dropwizard-mongodb	31,401	7,967	3,893	2,329	1,741	Ful	Jav	JAX	Jty	Lin	Mo	Lin	Ful	Rea	15,783
dropwizard-mysql	44,933	13,152	6,091	3,951	2,970	Ful	Jav	JAX	Jty	Lin	My	Lin	Ful	Rea	0
dropwizard-postgres	44,318	13,984	6,444	4,028	3,092	Ful	Jav	JAX	Jty	Lin	Pg	Lin	Ful	Rea	0
falcore	46,971	11,026	5,603	3,728	2,797	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
ffead-cpp	0	0	0	0	0	Ful	C++	ffe	ffe	Lin	Mo	Lin	Ful	Rea	0
ffead-cpp-apache-mon	0	0	0	0	0	Ful	C++	ffe	Apc	Lin	Mo	Lin	Ful	Rea	0
ffead-cpp-apache-mys	0	0	0	0	0	Ful	C++	ffe	Apc	Lin	My	Lin	Ful	Rea	0
ffead-cpp-apache-pos	6,883	1,455	730	489	363	Ful	C++	ffe	Apc	Lin	Pg	Lin	Ful	Rea	0
ffead-cpp-mysql	0	0	0	0	0	Ful	C++	ffe	ffe	Lin	My	Lin	Ful	Rea	0
ffead-cpp-nginx-mong	0	0	0	0	0	Ful	C++	ffe	ngx	Lin	Mo	Lin	Ful	Rea	0
ffead-cpp-nginx-mysq	0	0	0	0	0	Ful	C++	ffe	ngx	Lin	My	Lin	Ful	Rea	0
ffead-cpp-nginx-post	6,844	1,441	724	485	362	Ful	C++	ffe	ngx	Lin	Pg	Lin	Ful	Rea	0
ffead-cpp-postgresql	0	0	0	0	0	Ful	C++	ffe	ffe	Lin	Pg	Lin	Ful	Rea	0
fintrospect	90,612	19,843	10,192	6,827	5,168	Mcr	Sca	Nty	Non	Lin	My	Lin	Raw	Rea	0
flask	15,159	6,204	3,751	2,619	2,074	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
flask-mysql-raw	31,366	16,919	9,712	7,114	5,555	Mcr	Py	Mei	Non	Lin	My	Lin	Raw	Rea	0
flask-nginx-uwsgi	4,607	5,719	3,934	2,831	2,247	Mcr	Py	Non	ngx	Lin	My	Lin	Ful	Rea	0
flask-py3	17,180	7,118	4,253	3,135	2,485	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
flask-pypy2	1,321	557	334	276	274	Mcr	Py	Non	Tor	Lin	My	Lin	Ful	Rea	0
flask-pypy2-mysql-ra	2,592	1,667	1,400	1,598	1,506	Mcr	Py	Tor	Non	Lin	My	Lin	Raw	Rea	0
gemini-mysql	59,447	11,775	6,097	4,308	2,910	Ful	Jav	Svt	Res	Lin	My	Lin	Mcr	Rea	0
gemini-postgres	114,441	25,500	11,844	8,881	6,109	Ful	Jav	Svt	Res	Lin	Pg	Lin	Mcr	Rea	0
gin	46,464	11,343	5,477	3,846	2,870	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
goji	22,428	11,183	5,600	3,660	2,850	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
grizzly-jersey	72,616	15,534	7,769	5,147	3,878	Mcr	Jav	JAX	Non	Lin	My	Lin	Ful	Rea	0
http4s	61,885	19,991	10,625	7,112	5,366	Mcr	Sca	NIO	bla	Lin	Pg	Lin	Raw	Rea	0
jawn	117,531	31,120	16,199	10,980	8,332	Ful	Jav	Svt	Utw	Lin	Pg	Lin	Raw	Rea	0
undertow-jersey-c3p0	25,379	5,602	2,920	1,802	454	Plt	Jav	JAX	Non	Lin	My	Lin	Ful	Rea	0
undertow-jersey-hika	51,048	15,416	7,640	5,149	3,652	Plt	Jav	JAX	Non	Lin	My	Lin	Ful	Rea	0
jooby	92,811	36,144	18,118	12,207	9,196	Ful	Jav	Nty	Non	Lin	Pg	Lin	Raw	Rea	0
kami	84,211	20,420	9,947	6,512	4,938	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
klein	302	108	58	31	20	Mcr	Py	Non	Twi	Lin	My	Lin	Ful	Rea	443
minijax	43,814	11,866	6,017	4,111	3,135	Ful	Jav	JAX	Jty	Lin	My	Lin	Ful	Rea	0
morepath	44,489	15,881	8,313	5,652	4,247	Mcr	Py	Mei	Gun	Lin	Pg	Lin	Ful	Rea	0
ninja-standalone	8,985	7,449	6,498	4,838	3,629	Ful	Jav	Jty	Non	Lin	My	Lin	Ful	Rea	0
cherrypy	201	101	58	36	25	Mcr	Py	Non	Non	Lin	My	Lin	Ful	Rea	218
cherrypy-py3	275	124	73	48	31	Mcr	Py	Non	Non	Lin	My	Lin	Ful	Rea	366
cppcms	14,342	11,558	9,253	6,360	4,739	Plt	C++	Non	Non	Lin	My	Lin	Raw	Rea	0
cppcms-direct	10,604	9,075	7,726	6,375	4,707	Plt	C++	Non	Non	Lin	My	Lin	Raw	Rea	0
cppcms-postgres	14,856	11,806	9,149	5,105	2,782	Plt	C++	Non	Non	Lin	Pg	Lin	Raw	Rea	0
echo	37,228	7,895	4,226	2,995	2,247	Mcr	Go	Non	Non	Lin	Pg	Lin	Raw	Rea	0
fasthttp	38,448	8,314	4,532	3,127	2,350	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
fasthttp-postgresql	90,965	22,961	11,901	7,964	6,091	Plt	Go	Non	Non	Lin	Pg	Lin	Raw	Rea	0
go	37,841	8,191	4,385	3,011	2,279	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
go-interpolate	56,351	13,284	7,103	4,607	3,467	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
go-mongo	15,299	3,248	1,609	1,063	785	Plt	Go	Non	Non	Lin	Mo	Lin	Raw	Rea	0
go-postgres	38,216	7,848	4,364	2,946	2,265	Plt	Go	Non	Non	Lin	Pg	Lin	Raw	Rea	0
go-prefork	122,275	24,521	12,159	8,065	6,072	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
light-4j-postgresql	172,238	35,789	17,818	11,867	8,855	Plt	Jav	Lig	Non	Lin	Pg	Lin	Raw	Rea	0
revel-jet	45,487	10,161	5,076	3,403	2,580	Ful	Go	Non	Non	Lin	My	Lin	Mcr	Rea	0
revel-qbs	45,532	10,356	5,219	3,437	2,603	Ful	Go	Non	Non	Lin	My	Lin	Mcr	Rea	0
revel-raw	45,812	10,811	5,469	3,696	2,716	Ful	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
servlet-postgres-raw	51,886	24,890	12,600	8,439	6,341	Plt	Jav	Svt	Res	Lin	Pg	Lin	Raw	Rea	0
servlet-raw	50,550	20,376	10,279	6,849	5,100	Plt	Jav	Svt	Res	Lin	My	Lin	Raw	Rea	0
tornado	14,852	5,215	2,882	1,966	1,501	Plt	Py	Non	Tor	Lin	Mo	Lin	Raw	Rea	0
tornado-postgresql-r	7,404	4,201	2,584	733	376	Plt	Py	Non	Tor	Lin	Pg	Lin	Raw	Rea	911
tornado-py3	15,708	5,978	3,200	2,109	1,589	Plt	Py	Non	Tor	Lin	Mo	Lin	Raw	Rea	0
tornado-pypy2	45,109	12,404	6,788	4,567	3,438	Plt	Py	Non	Tor	Lin	Mo	Lin	Raw	Rea	0
undertow-mongodb	65,512	10,109	5,099	3,760	2,483	Plt	Jav	Utw	Non	Lin	Mo	Lin	Raw	Rea	10,091
undertow-mongodb-asy	54,470	12,012	5,993	4,006	2,904	Plt	Jav	Utw	Non	Lin	Mo	Lin	Raw	Rea	266
undertow-mysql	135,385	29,186	14,546	9,771	7,286	Plt	Jav	Utw	Non	Lin	My	Lin	Raw	Rea	0
undertow-postgresql	159,359	38,036	18,971	12,654	9,514	Plt	Jav	Utw	Non	Lin	Pg	Lin	Raw	Rea	0
vertx-postgres	187,903	33,862	16,877	11,368	8,697	Plt	Jav	Ver	Non	Lin	Pg	Lin	Raw	Rea	0
wildfly-ee7	63,853	23,123	11,066	7,597	5,292	Ful	Jav	Svt	Wil	Lin	My	Lin	Ful	Rea	0
play1	41,142	14,844	8,571	6,032	4,640	Ful	Jav	Nty	Non	Lin	My	Lin	Ful	Rea	0
play2-java-ebean-hik	20,336	4,937	2,520	1,694	1,272	Ful	Jav	Akk	Non	Lin	My	Lin	Ful	Rea	0
play2-java-ebean-hik	20,320	4,784	2,490	1,661	1,258	Ful	Jav	Nty	Non	Lin	My	Lin	Ful	Rea	0
play2-java-jooq-hika	22,470	7,217	2,823	2,506	1,241	Ful	Jav	Akk	Non	Lin	My	Lin	Ful	Rea	0
play2-java-jooq-hika	22,527	6,148	3,868	1,672	2,028	Ful	Jav	Nty	Non	Lin	My	Lin	Ful	Rea	0
play2-java-jpa-hikar	33,417	8,867	4,513	3,056	2,311	Ful	Jav	Akk	Non	Lin	My	Lin	Ful	Rea	0
play2-java-jpa-hikar	34,164	8,463	4,497	3,010	2,271	Ful	Jav	Nty	Non	Lin	My	Lin	Ful	Rea	0
play2-scala-anorm-li	39,957	11,116	5,706	3,728	2,826	Ful	Sca	Akk	Non	Lin	My	Lin	Ful	Rea	0
play2-scala-anorm-li	44,442	10,745	5,498	3,747	2,796	Ful	Sca	Nty	Non	Lin	My	Lin	Ful	Rea	0
play2-scala-reactive	14,317	3,799	2,007	1,382	967	Ful	Sca	Akk	Non	Lin	Mo	Lin	Ful	Rea	0
play2-scala-reactive	15,886	3,187	2,326	1,700	1,250	Ful	Sca	Nty	Non	Lin	Mo	Lin	Ful	Rea	0
play2-scala-slick	13,740	2,687	1,367	927	689	Ful	Sca	Akk	Non	Lin	My	Lin	Ful	Rea	247
play2-scala-slick-ne	13,395	2,679	1,336	889	651	Ful	Sca	Nty	Non	Lin	My	Lin	Ful	Rea	0
pyramid	22,667	6,260	3,371	2,266	1,733	Ful	Py	Non	Mei	Lin	Pg	Lin	Ful	Rea	0
pyramid-py2	19,263	5,420	2,885	1,966	1,449	Ful	Py	Non	Mei	Lin	Pg	Lin	Ful	Rea	0
restexpress	24,992	5,384	2,609	1,799	1,526	Mcr	Jav	Nty	Non	Lin	Mo	Lin	Raw	Rea	0
restexpress-mysql-ra	666	666	333	636	516	Mcr	Jav	Nty	Non	Lin	My	Lin	Raw	Rea	76,363
revenj-jvm	122,583	24,738	12,351	8,085	6,133	Ful	Jav	Svt	Res	Lin	Pg	Lin	Ful	Rea	0
silicon	60,571	20,719	11,411	7,880	6,004	Mcr	C++	Non	mic	Lin	My	Lin	Ful	Rea	53
spark	12,140	5,541	3,281	524	384	Mcr	Jav	Svt	Res	Lin	My	Lin	Ful	Rea	0
spring	30,386	6,183	3,090	2,064	1,516	Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	0
tapestry	31,131	11,015	6,111	4,143	3,204	Ful	Jav	Svt	Res	Lin	My	Lin	Ful	Rea	0
treefrog	45,358	9,018	4,497	2,997	2,216	Ful	C++	Non	Non	Lin	My	Lin	Mcr	Rea	0
treefrog-hybrid	51,310	10,149	5,032	3,334	2,497	Ful	C++	Non	Non	Lin	My	Lin	Mcr	Rea	2
treefrog-mongodb	133,735	26,975	13,434	8,908	6,688	Ful	C++	Non	Non	Lin	Mo	Lin	Mcr	Rea	0
treefrog-postgres	37,994	7,445	3,720	2,452	1,830	Ful	C++	Non	Non	Lin	Pg	Lin	Mcr	Rea	0
turbogears	14,382	5,189	2,960	2,052	1,593	Mcr	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
ulib-mongodb	174,524	35,067	17,539	11,676	8,745	Plt	C++	Non	ULi	Lin	Mo	Lin	Mcr	Rea	0
ulib-mysql	149,223	29,045	14,384	9,625	7,187	Plt	C++	Non	ULi	Lin	My	Lin	Mcr	Rea	0
ulib-postgres	179,923	35,362	17,525	11,660	8,699	Plt	C++	Non	ULi	Lin	Pg	Lin	Mcr	Rea	0
ulib-postgres_fit	185,382	37,108	18,440	12,194	9,171	Plt	C++	Non	ULi	Lin	Pg	Lin	Mcr	Rea	0
uvicorn	20,564	6,861	3,757	2,601	1,985	Plt	Py	Non	Non	Lin	Pg	Lin	Raw	Rea	0
vertx-web-mongodb	55,749	11,709	5,731	3,812	2,845	Mcr	Jav	vtx	Non	Lin	Mo	Lin	Raw	Rea	12
vertx-web-postgres	169,454	35,657	17,867	11,980	9,012	Mcr	Jav	vtx	Non	Lin	Pg	Lin	Raw	Rea	0
vertx-web-susom-post	112,205	33,214	17,698	12,097	9,130	Mcr	Jav	vtx	Non	Lin	Pg	Lin	Raw	Rea	0
web2py	8,016	5,512	4,072	3,234	2,699	Ful	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
web2py-optimized	14,806	10,658	7,590	5,937	4,712	Ful	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
webware	0	0	0	0	0	Mcr	Py	Non	Non	Lin	My	Lin	Ful	Rea	807
weppy	32,083	10,378	5,784	4,122	3,029	Ful	Py	Mei	Non	Lin	Pg	Lin	Ful	Rea	0
weppy-nginx-uwsgi	4,482	5,855	5,562	4,181	3,173	Ful	Py	uWS	ngx	Lin	Pg	Lin	Ful	Rea	0
weppy-py3	36,806	11,833	6,861	5,006	3,843	Ful	Py	Mei	Non	Lin	Pg	Lin	Ful	Rea	0
weppy-pypy2	4,404	2,328	1,875	1,781	1,585	Ful	Py	Tor	Non	Lin	Pg	Lin	Ful	Rea	0
wheezyweb	27,242	6,482	3,335	2,270	1,734	Mcr	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
wheezyweb-py3	— Did not complete —						Mcr	Py	Non	Mei	Lin	My	Lin	Ful	Rea	—
wicket	30,389	9,313	4,955	3,447	2,559	Ful	Jav	Svt	Res	Lin	My	Lin	Ful	Rea	0
wt	54,025	21,712	10,859	7,220	5,433	Ful	C++	Non	Non	Lin	My	Lin	Ful	Rea	0
wt-postgres	56,547	37,688	18,801	12,483	9,349	Ful	C++	Non	Non	Lin	Pg	Lin	Ful	Rea	0

Latency of 20-query responses, Dell servers at ServerCentral

Framework	Average latency (lower is better)		σ (SD)	Max	Errors
undertow-postgresql	53.5 ms	0.8%	33.9 ms	345.7 ms	0
jooby	54.2 ms	0.8%	12.5 ms	273.9 ms	0
vertx-web-susom-post	55.5 ms	0.9%	12.2 ms	173.1 ms	0
vertx-web-postgres	56.2 ms	0.9%	22.0 ms	210.4 ms	0
ulib-postgres_fit	56.2 ms	0.9%	23.3 ms	209.4 ms	0
light-4j-postgresql	57.7 ms	0.9%	3.8 ms	166.3 ms	0
vertx-postgres	58.0 ms	0.9%	34.6 ms	535.9 ms	0
ulib-mongodb	59.0 ms	0.9%	29.3 ms	172.3 ms	0
ulib-postgres	59.2 ms	0.9%	29.4 ms	218.6 ms	0
jawn	61.1 ms	0.9%	37.1 ms	403.1 ms	0
minijax	62.2 ms	1.0%	28.6 ms	688.6 ms	0
wt-postgres	63.2 ms	1.0%	101.5 ms	2050.0 ms	0
cutelyst-thread-post	66.8 ms	1.0%	32.1 ms	272.6 ms	0
cutelyst-nginx-postg	67.0 ms	1.0%	2.4 ms	82.6 ms	0
cutelyst-thread-post	67.3 ms	1.0%	32.7 ms	367.1 ms	0
cutelyst-pf-postgres	68.5 ms	1.1%	31.7 ms	274.7 ms	0
undertow-mysql	70.0 ms	1.1%	40.1 ms	463.5 ms	0
ulib-mysql	71.1 ms	1.1%	27.3 ms	181.7 ms	0
cpoll_cppsp-postgres	71.5 ms	1.1%	25.0 ms	602.9 ms	0
servlet-postgres-raw	77.8 ms	1.2%	24.9 ms	557.4 ms	0
go-prefork	79.4 ms	1.2%	34.1 ms	492.6 ms	0
revenj-jvm	80.4 ms	1.2%	25.1 ms	534.3 ms	0
fasthttp-postgresql	81.4 ms	1.3%	28.8 ms	367.3 ms	0
treefrog-mongodb	81.7 ms	1.3%	71.6 ms	1090.0 ms	0
akka-http	82.5 ms	1.3%	24.7 ms	501.6 ms	0
cpoll_cppsp-postgres	83.0 ms	1.3%	19.8 ms	724.3 ms	0
bottle-mysql-raw	84.2 ms	1.3%	27.1 ms	285.2 ms	0
flask-mysql-raw	86.7 ms	1.3%	39.7 ms	338.8 ms	0
cpoll_cppsp-raw	87.5 ms	1.4%	69.3 ms	1340.0 ms	0
http4s	89.6 ms	1.4%	42.1 ms	852.9 ms	0
aiohttp-pg-raw	91.0 ms	1.4%	27.4 ms	740.4 ms	0
fintrospect	95.2 ms	1.5%	46.7 ms	899.0 ms	0
cutelyst-thread-mysq	96.0 ms	1.5%	43.7 ms	442.1 ms	0
silicon	96.6 ms	1.5%	141.2 ms	1790.0 ms	53
servlet-raw	96.9 ms	1.5%	22.3 ms	384.9 ms	0
cutelyst-thread-mysq	98.2 ms	1.5%	50.6 ms	522.6 ms	0
wildfly-ee7	99.1 ms	1.5%	46.6 ms	722.2 ms	0
kami	99.4 ms	1.5%	27.4 ms	586.7 ms	0
api_hour-mysql	101.3 ms	1.6%	66.7 ms	1040.0 ms	0
actframework-hiberna	103.2 ms	1.6%	15.7 ms	269.8 ms	0
web2py-optimized	104.0 ms	1.6%	45.9 ms	448.7 ms	0
play1	104.1 ms	1.6%	37.1 ms	900.3 ms	0
cppcms	108.3 ms	1.7%	21.5 ms	866.1 ms	0
cppcms-direct	108.5 ms	1.7%	14.9 ms	364.6 ms	0
cutelyst-nginx-mysql	108.7 ms	1.7%	5.0 ms	126.7 ms	0
actframework-morphia	109.2 ms	1.7%	24.4 ms	242.8 ms	0
cutelyst-pf-mysql	110.7 ms	1.7%	48.4 ms	386.3 ms	0
actframework-ebean-p	110.9 ms	1.7%	20.2 ms	329.6 ms	0
morepath	117.1 ms	1.8%	38.1 ms	400.0 ms	0
wt	120.4 ms	1.9%	183.6 ms	2830.0 ms	0
actframework-hiberna	120.5 ms	1.9%	13.1 ms	334.4 ms	0
weppy-py3	125.0 ms	1.9%	47.5 ms	393.5 ms	0
actframework-ebean-m	126.1 ms	2.0%	17.8 ms	403.3 ms	0
grizzly-jersey	126.6 ms	2.0%	21.0 ms	340.5 ms	0
undertow-jersey-hika	135.8 ms	2.1%	60.7 ms	570.3 ms	0
ninja-standalone	139.0 ms	2.2%	65.1 ms	1830.0 ms	0
cutelyst-pf-mysql-ep	139.6 ms	2.2%	146.0 ms	1830.0 ms	0
tornado-pypy2	144.1 ms	2.2%	76.9 ms	748.1 ms	0
go-interpolate	147.0 ms	2.3%	22.7 ms	244.1 ms	0
tapestry	152.9 ms	2.4%	34.0 ms	537.5 ms	0
weppy	158.9 ms	2.5%	69.9 ms	627.5 ms	0
dropwizard-postgres	159.7 ms	2.5%	46.3 ms	424.8 ms	0
weppy-nginx-uwsgi	160.6 ms	2.5%	26.5 ms	473.4 ms	0
dropwizard-mysql	163.0 ms	2.5%	50.0 ms	433.3 ms	0
play2-scala-anorm-li	170.6 ms	2.6%	36.2 ms	712.2 ms	0
gin	172.3 ms	2.7%	47.2 ms	491.1 ms	0
dropwizard-mongodb	174.8 ms	2.7%	101.2 ms	491.3 ms	15,783
goji	178.5 ms	2.8%	43.5 ms	390.6 ms	0
vertx-web-mongodb	180.0 ms	2.8%	85.7 ms	682.2 ms	12
api_hour	180.3 ms	2.8%	184.2 ms	1970.0 ms	0
revel-raw	180.4 ms	2.8%	51.8 ms	514.6 ms	0
web2py	181.6 ms	2.8%	97.8 ms	852.5 ms	0
undertow-mongodb	181.8 ms	2.8%	155.0 ms	798.2 ms	10,091
falcore	181.9 ms	2.8%	43.7 ms	385.9 ms	0
play2-scala-anorm-li	182.6 ms	2.8%	27.5 ms	504.9 ms	0
revel-qbs	183.7 ms	2.8%	52.6 ms	474.6 ms	0
undertow-mongodb-asy	189.7 ms	2.9%	175.9 ms	637.4 ms	266
revel-jet	189.9 ms	2.9%	53.1 ms	523.8 ms	0
treefrog-hybrid	193.1 ms	3.0%	36.0 ms	657.5 ms	2
wicket	193.4 ms	3.0%	76.5 ms	819.6 ms	0
activeweb-jackson	202.6 ms	3.1%	37.8 ms	606.7 ms	0
flask-py3	203.7 ms	3.2%	118.3 ms	1110.0 ms	0
aiohttp	210.0 ms	3.3%	95.0 ms	783.6 ms	0
fasthttp	210.4 ms	3.3%	52.2 ms	691.8 ms	0
play2-java-jpa-hikar	215.4 ms	3.3%	38.8 ms	597.7 ms	0
activeweb	217.4 ms	3.4%	47.0 ms	496.1 ms	0
treefrog	221.7 ms	3.4%	42.7 ms	706.3 ms	0
cppcms-postgres	222.4 ms	3.4%	200.8 ms	1430.0 ms	0
go	222.9 ms	3.5%	35.8 ms	390.1 ms	0
gemini-mysql	223.4 ms	3.5%	245.7 ms	2330.0 ms	0
play2-java-jpa-hikar	223.9 ms	3.5%	24.4 ms	534.0 ms	0
go-postgres	224.4 ms	3.5%	39.8 ms	515.3 ms	0
flask-nginx-uwsgi	226.0 ms	3.5%	43.0 ms	491.5 ms	0
echo	226.1 ms	3.5%	42.1 ms	544.7 ms	0
aah	228.6 ms	3.5%	51.0 ms	718.7 ms	0
django-py3	241.3 ms	3.7%	131.1 ms	871.5 ms	0
flask	243.2 ms	3.8%	139.1 ms	1160.0 ms	0
uvicorn	245.3 ms	3.8%	143.9 ms	1200.0 ms	0
play2-scala-slick	248.0 ms	3.8%	541.7 ms	7890.0 ms	247
play2-java-jooq-hika	250.4 ms	3.9%	26.7 ms	550.0 ms	0
treefrog-postgres	266.6 ms	4.1%	71.9 ms	1090.0 ms	0
bottle-nginx-uwsgi	275.7 ms	4.3%	47.5 ms	746.9 ms	0
django-postgresql	286.1 ms	4.4%	146.0 ms	867.3 ms	0
pyramid	286.6 ms	4.4%	159.1 ms	1240.0 ms	0
wheezyweb	287.0 ms	4.4%	167.7 ms	1220.0 ms	0
bottle-py3	287.7 ms	4.5%	169.8 ms	1470.0 ms	0
turbogears	294.4 ms	4.6%	135.9 ms	973.7 ms	0
django	296.4 ms	4.6%	141.3 ms	1070.0 ms	0
bottle	301.6 ms	4.7%	157.2 ms	1400.0 ms	0
restexpress	308.0 ms	4.8%	117.6 ms	1370.0 ms	0
tornado-py3	314.1 ms	4.9%	160.2 ms	1330.0 ms	0
cherrypy-py3	314.6 ms	4.9%	475.2 ms	3750.0 ms	366
dropwizard-jdbi-post	323.3 ms	5.0%	396.7 ms	2310.0 ms	102
pyramid-py2	324.0 ms	5.0%	173.9 ms	1300.0 ms	0
spring	324.1 ms	5.0%	49.4 ms	527.8 ms	0
tornado	331.4 ms	5.1%	91.3 ms	911.3 ms	0
restexpress-mysql-ra	332.4 ms	5.1%	594.4 ms	3660.0 ms	76,363
cherrypy	379.7 ms	5.9%	554.6 ms	4050.0 ms	218
flask-pypy2-mysql-ra	379.9 ms	5.9%	416.4 ms	6210.0 ms	0
play2-java-ebean-hik	388.9 ms	6.0%	64.8 ms	1120.0 ms	0
play2-java-ebean-hik	392.1 ms	6.1%	47.4 ms	534.7 ms	0
gemini-postgres	431.6 ms	6.7%	878.0 ms	7990.0 ms	0
play2-scala-reactive	433.1 ms	6.7%	296.3 ms	1690.0 ms	0
blade	441.5 ms	6.8%	141.7 ms	1180.0 ms	0
weppy-pypy2	452.0 ms	7.0%	614.4 ms	7560.0 ms	0
beego-orm-mysql	463.2 ms	7.2%	100.2 ms	1230.0 ms	0
play2-scala-reactive	492.6 ms	7.6%	349.7 ms	2170.0 ms	0
go-mongo	630.6 ms	9.8%	3.5 ms	652.8 ms	0
play2-scala-slick-ne	920.7 ms	14.3%	1250.0 ms	7810.0 ms	0
play2-java-jooq-hika	962.6 ms	14.9%	1420.0 ms	6140.0 ms	0
undertow-jersey-c3p0	1140.0 ms	17.6%	1070.0 ms	7220.0 ms	0
tornado-postgresql-r	1290.0 ms	20.0%	1320.0 ms	5190.0 ms	911
ffead-cpp-apache-pos	1300.0 ms	20.1%	250.9 ms	1510.0 ms	0
ffead-cpp-nginx-post	1350.0 ms	20.9%	208.0 ms	1550.0 ms	0
spark	1480.0 ms	22.9%	1460.0 ms	7510.0 ms	0
flask-pypy2	1540.0 ms	23.8%	1210.0 ms	7990.0 ms	0
bottle-pypy2	1580.0 ms	24.5%	1100.0 ms	8000.0 ms	0
klein	5410.0 ms	83.7%	1200.0 ms	7060.0 ms	443
webware	6460.0 ms	100.0%	678.1 ms	7990.0 ms	807

Framework overhead of 20-query responses, Dell servers at ServerCentral

Framework best		vs	Platform best		Ratio (higher is better)
undertow-jersey-hika	3,652	vs	undertow-jersey-c3p0	454	804.4%
treefrog-mongodb	6,688	vs	treefrog	2,216	301.8%
fasthttp-postgresql	6,091	vs	go	2,279	267.3%
go-prefork	6,072	vs	go	2,279	266.4%
tornado-pypy2	3,438	vs	tornado	1,501	229.0%
kami	4,938	vs	go	2,279	216.7%
wt-postgres	9,349	vs	wt	5,433	172.1%
go-interpolate	3,467	vs	go	2,279	152.1%
gin	2,870	vs	go	2,279	125.9%
goji	2,850	vs	go	2,279	125.1%
servlet-postgres-raw	6,341	vs	servlet-raw	5,100	124.3%
falcore	2,797	vs	go	2,279	122.7%
revel-raw	2,716	vs	go	2,279	119.2%
revel-qbs	2,603	vs	go	2,279	114.2%
revel-jet	2,580	vs	go	2,279	113.2%
treefrog-hybrid	2,497	vs	treefrog	2,216	112.7%
tornado-py3	1,589	vs	tornado	1,501	105.9%
fasthttp	2,350	vs	go	2,279	103.1%
go-postgres	2,265	vs	go	2,279	99.4%
echo	2,247	vs	go	2,279	98.6%
aah	2,221	vs	go	2,279	97.5%
dropwizard-mysql	2,970	vs	servlet-raw	5,100	58.2%
dropwizard-postgres	3,092	vs	servlet-postgres-raw	6,341	48.8%
play2-scala-anorm-li	2,826	vs	akka-http	5,897	47.9%
beego-orm-mysql	1,087	vs	go	2,279	47.7%
play2-java-jpa-hikar	2,311	vs	akka-http	5,897	39.2%
go-mongo	785	vs	go	2,279	34.4%
dropwizard-jdbi-post	2,108	vs	servlet-postgres-raw	6,341	33.2%
tornado-postgresql-r	376	vs	tornado	1,501	25.0%
play2-java-ebean-hik	1,272	vs	akka-http	5,897	21.6%
play2-java-jooq-hika	1,241	vs	akka-http	5,897	21.0%
play2-scala-reactive	967	vs	akka-http	5,897	16.4%
undertow-jersey-c3p0	454	vs	undertow-jersey-hika	3,652	12.4%
play2-scala-slick	689	vs	akka-http	5,897	11.7%

20-queries (bar)

Data table

Latency

Framework overhead

20-queries (bar)

Data table

Latency

Framework overhead

Requirements summary

In this test, each request is processed by fetching multiple rows from a simple database table and serializing these rows as a JSON response. The test is run multiple times: testing 1, 5, 10, 15, and 20 queries per request. All tests are run at 256 concurrency.

Example response for 10 queries:

HTTP/1.1 200 OK Content-Length: 315 Content-Type: application/json Server: Example Date: Wed, 17 Apr 2013 12:00:00 GMT [{"id":4174,"randomNumber":331},{"id":51,"randomNumber":6544},{"id":4462,"randomNumber":952},{"id":2221,"randomNumber":532},{"id":9276,"randomNumber":3097},{"id":3056,"randomNumber":7293},{"id":6964,"randomNumber":620},{"id":675,"randomNumber":6601},{"id":8414,"randomNumber":6569},{"id":2753,"randomNumber":4065}]

For a more detailed description of the requirements, see the Source Code and Requirements section.

Responses per second at 20 updates per request, Dell servers at ServerCentral(138 tests)

Framework	Performance (higher is better)		Cls	Lng	Plt	FE	Aos	DB	Dos	Orm	IA	Errors
ulib-postgres_fit	4,886	100.0%(95.1%)	Plt	C++	Non	ULi	Lin	Pg	Lin	Mcr	Rea	0
vertx-web-postgres	4,002	81.9%(77.9%)	Mcr	Jav	vtx	Non	Lin	Pg	Lin	Raw	Rea	0
ulib-postgres	3,729	76.3%(72.6%)	Plt	C++	Non	ULi	Lin	Pg	Lin	Mcr	Rea	0
vertx-postgres	3,478	71.2%(67.7%)	Plt	Jav	Ver	Non	Lin	Pg	Lin	Raw	Rea	0
undertow-postgresql	3,352	68.6%(65.3%)	Plt	Jav	Utw	Non	Lin	Pg	Lin	Raw	Rea	0
ulib-mongodb	3,313	67.8%(64.5%)	Plt	C++	Non	ULi	Lin	Mo	Lin	Mcr	Rea	0
jawn	3,252	66.6%(63.3%)	Ful	Jav	Svt	Utw	Lin	Pg	Lin	Raw	Rea	0
dropwizard-postgres	3,243	66.4%(63.1%)	Ful	Jav	JAX	Jty	Lin	Pg	Lin	Ful	Rea	0
tornado-pypy2	3,185	65.2%(62.0%)	Plt	Py	Non	Tor	Lin	Mo	Lin	Raw	Rea	0
cutelyst-nginx-postg	3,073	62.9%(59.8%)	Ful	C++	Qt	ngx	Lin	Pg	Lin	Raw	Rea	0
cutelyst-pf-postgres	3,057	62.6%(59.5%)	Ful	C++	Qt	Non	Lin	Pg	Lin	Raw	Rea	0
wt-postgres	2,975	60.9%(57.9%)	Ful	C++	Non	Non	Lin	Pg	Lin	Ful	Rea	6
undertow-mongodb-asy	2,873	58.8%(55.9%)	Plt	Jav	Utw	Non	Lin	Mo	Lin	Raw	Rea	18
dropwizard-mysql	2,820	57.7%(54.9%)	Ful	Jav	JAX	Jty	Lin	My	Lin	Ful	Rea	0
cutelyst-thread-post	2,801	57.3%(54.5%)	Ful	C++	Qt	Non	Lin	Pg	Lin	Raw	Rea	0
cutelyst-thread-post	2,794	57.2%(54.4%)	Ful	C++	Qt	Non	Lin	Pg	Lin	Raw	Rea	0
fasthttp-postgresql	2,702	55.3%(52.6%)	Plt	Go	Non	Non	Lin	Pg	Lin	Raw	Rea	0
treefrog-mongodb	2,608	53.4%(50.8%)	Ful	C++	Non	Non	Lin	Mo	Lin	Mcr	Rea	0
ulib-mysql	2,468	50.5%(48.1%)	Plt	C++	Non	ULi	Lin	My	Lin	Mcr	Rea	0
undertow-mysql	2,453	50.2%(47.8%)	Plt	Jav	Utw	Non	Lin	My	Lin	Raw	Rea	0
undertow-mongodb	2,404	49.2%(46.8%)	Plt	Jav	Utw	Non	Lin	Mo	Lin	Raw	Rea	10,745
actframework-morphia	2,285	46.8%(44.5%)	Ful	Jav	Utw	Non	Lin	Mo	Lin	Ful	Rea	0
jooby	2,264	46.3%(44.1%)	Ful	Jav	Nty	Non	Lin	Pg	Lin	Raw	Rea	0
revenj-jvm	2,255	46.2%(43.9%)	Ful	Jav	Svt	Res	Lin	Pg	Lin	Ful	Rea	0
fintrospect	2,241	45.9%(43.6%)	Mcr	Sca	Nty	Non	Lin	My	Lin	Raw	Rea	0
minijax	2,182	44.7%(42.5%)	Ful	Jav	JAX	Jty	Lin	My	Lin	Ful	Rea	0
akka-http	2,117	43.3%(41.2%)	Mcr	Sca	Akk	Non	Lin	My	Lin	Raw	Rea	0
light-4j-postgresql	2,058	42.1%(40.1%)	Plt	Jav	Lig	Non	Lin	Pg	Lin	Raw	Rea	0
cutelyst-thread-mysq	2,054	42.0%(40.0%)	Ful	C++	Qt	Non	Lin	My	Lin	Raw	Rea	0
wt	2,040	41.8%(39.7%)	Ful	C++	Non	Non	Lin	My	Lin	Ful	Rea	45
silicon	2,038	41.7%(39.7%)	Mcr	C++	Non	mic	Lin	My	Lin	Ful	Rea	0
cpoll_cppsp-raw	2,034	41.6%(39.6%)	Plt	C++	Non	Non	Lin	My	Lin	Raw	Rea	0
cutelyst-thread-mysq	2,016	41.3%(39.3%)	Ful	C++	Qt	Non	Lin	My	Lin	Raw	Rea	0
go-interpolate	1,913	39.2%(37.2%)	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
servlet-raw	1,850	37.9%(36.0%)	Plt	Jav	Svt	Res	Lin	My	Lin	Raw	Rea	0
flask-mysql-raw	1,846	37.8%(35.9%)	Mcr	Py	Mei	Non	Lin	My	Lin	Raw	Rea	0
fasthttp	1,840	37.7%(35.8%)	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
cutelyst-nginx-mysql	1,831	37.5%(35.7%)	Ful	C++	Qt	ngx	Lin	My	Lin	Raw	Rea	0
cutelyst-pf-mysql	1,828	37.4%(35.6%)	Ful	C++	Qt	Non	Lin	My	Lin	Raw	Rea	0
cutelyst-pf-mysql-ep	1,823	37.3%(35.5%)	Ful	C++	Qt	Non	Lin	My	Lin	Raw	Rea	0
bottle-mysql-raw	1,804	36.9%(35.1%)	Mcr	Py	Mei	Non	Lin	My	Lin	Raw	Rea	0
morepath	1,789	36.6%(34.8%)	Mcr	Py	Mei	Gun	Lin	Pg	Lin	Ful	Rea	0
actframework-ebean-m	1,772	36.3%(34.5%)	Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	0
dropwizard-jdbi-post	1,749	35.8%(34.1%)	Ful	Jav	JAX	Jty	Lin	Pg	Lin	Mcr	Rea	0
actframework-hiberna	1,723	35.3%(33.5%)	Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	0
actframework-ebean-p	1,694	34.7%(33.0%)	Ful	Jav	Utw	Non	Lin	Pg	Lin	Ful	Rea	0
actframework-hiberna	1,680	34.4%(32.7%)	Ful	Jav	Utw	Non	Lin	Pg	Lin	Ful	Rea	0
falcore	1,650	33.8%(32.1%)	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
gin	1,649	33.7%(32.1%)	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
wildfly-ee7	1,633	33.4%(31.8%)	Ful	Jav	Svt	Wil	Lin	My	Lin	Ful	Rea	0
cppcms-postgres	1,589	32.5%(30.9%)	Plt	C++	Non	Non	Lin	Pg	Lin	Raw	Rea	0
weppy-py3	1,483	30.4%(28.9%)	Ful	Py	Mei	Non	Lin	Pg	Lin	Ful	Rea	0
go-prefork	1,481	30.3%(28.8%)	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
play2-scala-anorm-li	1,480	30.3%(28.8%)	Ful	Sca	Akk	Non	Lin	My	Lin	Ful	Rea	0
play2-scala-anorm-li	1,474	30.2%(28.7%)	Ful	Sca	Nty	Non	Lin	My	Lin	Ful	Rea	0
servlet-postgres-raw	1,471	30.1%(28.6%)	Plt	Jav	Svt	Res	Lin	Pg	Lin	Raw	Rea	0
revel-raw	1,467	30.0%(28.6%)	Ful	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
vertx-web-mongodb	1,433	29.3%(27.9%)	Mcr	Jav	vtx	Non	Lin	Mo	Lin	Raw	Rea	23
go	1,422	29.1%(27.7%)	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
go-postgres	1,412	28.9%(27.5%)	Plt	Go	Non	Non	Lin	Pg	Lin	Raw	Rea	0
pyramid	1,407	28.8%(27.4%)	Ful	Py	Non	Mei	Lin	Pg	Lin	Ful	Rea	0
aah	1,406	28.8%(27.4%)	Ful	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
tornado	1,398	28.6%(27.2%)	Plt	Py	Non	Tor	Lin	Mo	Lin	Raw	Rea	0
goji	1,395	28.6%(27.2%)	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
echo	1,381	28.3%(26.9%)	Mcr	Go	Non	Non	Lin	Pg	Lin	Raw	Rea	0
weppy	1,352	27.7%(26.3%)	Ful	Py	Mei	Non	Lin	Pg	Lin	Ful	Rea	0
revel-jet	1,272	26.0%(24.8%)	Ful	Go	Non	Non	Lin	My	Lin	Mcr	Rea	0
flask-pypy2-mysql-ra	1,270	26.0%(24.7%)	Mcr	Py	Tor	Non	Lin	My	Lin	Raw	Rea	0
play2-java-jooq-hika	1,214	24.8%(23.6%)	Ful	Jav	Nty	Non	Lin	My	Lin	Ful	Rea	0
pyramid-py2	1,209	24.7%(23.5%)	Ful	Py	Non	Mei	Lin	Pg	Lin	Ful	Rea	0
kami	1,185	24.3%(23.1%)	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
weppy-nginx-uwsgi	1,181	24.2%(23.0%)	Ful	Py	uWS	ngx	Lin	Pg	Lin	Ful	Rea	0
play2-java-jooq-hika	1,171	24.0%(22.8%)	Ful	Jav	Akk	Non	Lin	My	Lin	Ful	Rea	0
wicket	1,165	23.8%(22.7%)	Ful	Jav	Svt	Res	Lin	My	Lin	Ful	Rea	0
flask-py3	1,155	23.6%(22.5%)	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
activeweb-jackson	1,119	22.9%(21.8%)	Ful	Jav	Svt	Non	Lin	My	Lin	Mcr	Rea	0
cppcms	1,115	22.8%(21.7%)	Plt	C++	Non	Non	Lin	My	Lin	Raw	Rea	0
play2-java-jpa-hikar	1,110	22.7%(21.6%)	Ful	Jav	Akk	Non	Lin	My	Lin	Ful	Rea	13
cppcms-direct	1,109	22.7%(21.6%)	Plt	C++	Non	Non	Lin	My	Lin	Raw	Rea	0
gemini-postgres	1,102	22.6%(21.5%)	Ful	Jav	Svt	Res	Lin	Pg	Lin	Mcr	Rea	0
play2-java-jpa-hikar	1,087	22.2%(21.2%)	Ful	Jav	Nty	Non	Lin	My	Lin	Ful	Rea	11
flask	1,077	22.0%(21.0%)	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
api_hour	1,041	21.3%(20.3%)	Mcr	Py	asy	Gun	Lin	Pg	Lin	Raw	Rea	0
activeweb	1,010	20.7%(19.7%)	Ful	Jav	Svt	Non	Lin	My	Lin	Mcr	Rea	0
revel-qbs	1,003	20.5%(19.5%)	Ful	Go	Non	Non	Lin	My	Lin	Mcr	Rea	0
aiohttp-pg-raw	974	19.9%(19.0%)	Mcr	Py	asy	Gun	Lin	Pg	Lin	Raw	Rea	0
play2-java-ebean-hik	966	19.8%(18.8%)	Ful	Jav	Nty	Non	Lin	My	Lin	Ful	Rea	7
play2-java-ebean-hik	956	19.6%(18.6%)	Ful	Jav	Akk	Non	Lin	My	Lin	Ful	Rea	10
turbogears	950	19.4%(18.5%)	Mcr	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
tornado-py3	940	19.2%(18.3%)	Plt	Py	Non	Tor	Lin	Mo	Lin	Raw	Rea	0
bottle-py3	935	19.1%(18.2%)	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
treefrog-hybrid	927	19.0%(18.0%)	Ful	C++	Non	Non	Lin	My	Lin	Mcr	Rea	1
bottle	919	18.8%(17.9%)	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
aiohttp	890	18.2%(17.3%)	Mcr	Py	asy	Gun	Lin	Pg	Lin	Ful	Rea	0
flask-nginx-uwsgi	870	17.8%(16.9%)	Mcr	Py	Non	ngx	Lin	My	Lin	Ful	Rea	0
dropwizard-mongodb	868	17.8%(16.9%)	Ful	Jav	JAX	Jty	Lin	Mo	Lin	Ful	Rea	17,055
django-py3	857	17.5%(16.7%)	Ful	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
vertx-web-susom-post	828	16.9%(16.1%)	Mcr	Jav	vtx	Non	Lin	Pg	Lin	Raw	Rea	0
gemini-mysql	819	16.8%(15.9%)	Ful	Jav	Svt	Res	Lin	My	Lin	Mcr	Rea	0
beego-orm-mysql	801	16.4%(15.6%)	Ful	Go	Non	Non	Lin	My	Lin	Mcr	Rea	0
wheezyweb	777	15.9%(15.1%)	Mcr	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
uvicorn	760	15.6%(14.8%)	Plt	Py	Non	Non	Lin	Pg	Lin	Raw	Rea	0
bottle-nginx-uwsgi	737	15.1%(14.3%)	Mcr	Py	uWS	ngx	Lin	My	Lin	Ful	Rea	0
django-postgresql	710	14.5%(13.8%)	Ful	Py	Non	Mei	Lin	Pg	Lin	Ful	Rea	0
treefrog	703	14.4%(13.7%)	Ful	C++	Non	Non	Lin	My	Lin	Mcr	Rea	0
django	683	14.0%(13.3%)	Ful	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
spring	659	13.5%(12.8%)	Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	0
ninja-standalone	653	13.4%(12.7%)	Ful	Jav	Jty	Non	Lin	My	Lin	Ful	Rea	0
treefrog-postgres	568	11.6%(11.1%)	Ful	C++	Non	Non	Lin	Pg	Lin	Mcr	Rea	0
web2py-optimized	513	10.5%(10.0%)	Ful	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
play2-scala-slick	360	7.4%(7.0%)	Ful	Sca	Akk	Non	Lin	My	Lin	Ful	Rea	99
play2-scala-slick-ne	354	7.2%(6.9%)	Ful	Sca	Nty	Non	Lin	My	Lin	Ful	Rea	0
go-mongo	294	6.0%(5.7%)	Plt	Go	Non	Non	Lin	Mo	Lin	Raw	Rea	0
ffead-cpp-apache-pos	207	4.2%(4.0%)	Ful	C++	ffe	Apc	Lin	Pg	Lin	Ful	Rea	0
ffead-cpp-nginx-post	207	4.2%(4.0%)	Ful	C++	ffe	ngx	Lin	Pg	Lin	Ful	Rea	0
http4s	116	2.4%(2.3%)	Mcr	Sca	NIO	bla	Lin	Pg	Lin	Raw	Rea	35
web2py	107	2.2%(2.1%)	Ful	Py	Mei	Non	Lin	My	Lin	Ful	Rea	1,055
bottle-pypy2	95	1.9%(1.8%)	Mcr	Py	Tor	Non	Lin	My	Lin	Ful	Rea	0
flask-pypy2	90	1.8%(1.8%)	Mcr	Py	Non	Tor	Lin	My	Lin	Ful	Rea	0
spark	85	1.7%(1.7%)	Mcr	Jav	Svt	Res	Lin	My	Lin	Ful	Rea	0
play2-scala-reactive	56	1.1%(1.1%)	Ful	Sca	Akk	Non	Lin	Mo	Lin	Ful	Rea	0
play2-scala-reactive	48	1.0%(0.9%)	Ful	Sca	Nty	Non	Lin	Mo	Lin	Ful	Rea	0
cherrypy-py3	15	0.3%(0.3%)	Mcr	Py	Non	Non	Lin	My	Lin	Ful	Rea	192
weppy-pypy2	11	0.2%(0.2%)	Ful	Py	Tor	Non	Lin	Pg	Lin	Ful	Rea	142
klein	8	0.2%(0.2%)	Mcr	Py	Non	Twi	Lin	My	Lin	Ful	Rea	40
cherrypy	8	0.2%(0.2%)	Mcr	Py	Non	Non	Lin	My	Lin	Ful	Rea	330
api_hour-mysql	0	0.0%(0.0%)	Mcr	Py	asy	Gun	Lin	My	Lin	Raw	Rea	188
ffead-cpp	0	0.0%(0.0%)	Ful	C++	ffe	ffe	Lin	Mo	Lin	Ful	Rea	0
ffead-cpp-apache-mon	0	0.0%(0.0%)	Ful	C++	ffe	Apc	Lin	Mo	Lin	Ful	Rea	0
ffead-cpp-apache-mys	0	0.0%(0.0%)	Ful	C++	ffe	Apc	Lin	My	Lin	Ful	Rea	0
ffead-cpp-mysql	0	0.0%(0.0%)	Ful	C++	ffe	ffe	Lin	My	Lin	Ful	Rea	0
ffead-cpp-nginx-mong	0	0.0%(0.0%)	Ful	C++	ffe	ngx	Lin	Mo	Lin	Ful	Rea	0
ffead-cpp-nginx-mysq	0	0.0%(0.0%)	Ful	C++	ffe	ngx	Lin	My	Lin	Ful	Rea	0
ffead-cpp-postgresql	0	0.0%(0.0%)	Ful	C++	ffe	ffe	Lin	Pg	Lin	Ful	Rea	0
webware	0	0.0%(0.0%)	Mcr	Py	Non	Non	Lin	My	Lin	Ful	Rea	795
actframework-eclipse	—	Did not complete	Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	—
actframework-eclipse	—	Did not complete	Ful	Jav	Utw	Non	Lin	Pg	Lin	Ful	Rea	—
wheezyweb-py3	—	Did not complete	Mcr	Py	Non	Mei	Lin	My	Lin	Ful	Rea	—

Responses per second, multiple updates, Dell servers at ServerCentral

Framework	1	5	10	15	20	Cls	Lng	Plt	FE	Aos	DB	Dos	Orm	IA	Errors
aah	14,262	4,696	2,636	1,838	1,406	Ful	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
actframework-ebean-m	35,298	7,099	3,565	2,366	1,772	Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	0
actframework-ebean-p	33,126	6,735	3,387	2,266	1,694	Ful	Jav	Utw	Non	Lin	Pg	Lin	Ful	Rea	0
actframework-hiberna	35,664	7,123	3,485	2,334	1,723	Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	0
actframework-hiberna	32,430	6,689	3,389	2,250	1,680	Ful	Jav	Utw	Non	Lin	Pg	Lin	Ful	Rea	0
actframework-morphia	45,936	9,112	4,553	3,061	2,285	Ful	Jav	Utw	Non	Lin	Mo	Lin	Ful	Rea	0
actframework-eclipse	— Did not complete —						Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	—
actframework-eclipse	— Did not complete —						Ful	Jav	Utw	Non	Lin	Pg	Lin	Ful	Rea	—
activeweb	12,984	3,763	1,975	1,333	1,010	Ful	Jav	Svt	Non	Lin	My	Lin	Mcr	Rea	0
activeweb-jackson	23,162	4,662	2,328	1,552	1,119	Ful	Jav	Svt	Non	Lin	My	Lin	Mcr	Rea	0
aiohttp	17,328	3,682	1,799	1,201	890	Mcr	Py	asy	Gun	Lin	Pg	Lin	Ful	Rea	0
aiohttp-pg-raw	10,822	3,260	1,774	1,258	974	Mcr	Py	asy	Gun	Lin	Pg	Lin	Raw	Rea	0
akka-http	42,710	8,374	4,225	2,828	2,117	Mcr	Sca	Akk	Non	Lin	My	Lin	Raw	Rea	0
api_hour	16,576	3,574	1,857	1,289	1,041	Mcr	Py	asy	Gun	Lin	Pg	Lin	Raw	Rea	0
api_hour-mysql	9,213	200	11	0	0	Mcr	Py	asy	Gun	Lin	My	Lin	Raw	Rea	188
beego-orm-mysql	17,570	3,491	1,689	1,095	801	Ful	Go	Non	Non	Lin	My	Lin	Mcr	Rea	0
bottle	11,585	3,153	1,611	1,232	919	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
bottle-mysql-raw	37,257	7,366	3,687	2,441	1,804	Mcr	Py	Mei	Non	Lin	My	Lin	Raw	Rea	0
bottle-nginx-uwsgi	5,124	3,471	1,770	1,093	737	Mcr	Py	uWS	ngx	Lin	My	Lin	Ful	Rea	0
bottle-py3	12,350	3,406	1,872	1,351	935	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
bottle-pypy2	1,310	365	194	137	95	Mcr	Py	Tor	Non	Lin	My	Lin	Ful	Rea	0
cpoll_cppsp-raw	40,663	7,942	4,035	2,644	2,034	Plt	C++	Non	Non	Lin	My	Lin	Raw	Rea	0
cutelyst-nginx-mysql	34,772	7,359	3,703	2,473	1,831	Ful	C++	Qt	ngx	Lin	My	Lin	Raw	Rea	0
cutelyst-nginx-postg	57,133	12,164	6,120	4,090	3,073	Ful	C++	Qt	ngx	Lin	Pg	Lin	Raw	Rea	0
cutelyst-pf-mysql	35,484	7,389	3,708	2,403	1,828	Ful	C++	Qt	Non	Lin	My	Lin	Raw	Rea	0
cutelyst-pf-mysql-ep	31,845	7,223	3,696	2,433	1,823	Ful	C++	Qt	Non	Lin	My	Lin	Raw	Rea	0
cutelyst-pf-postgres	58,328	12,156	6,103	4,080	3,057	Ful	C++	Qt	Non	Lin	Pg	Lin	Raw	Rea	0
cutelyst-thread-mysq	41,640	8,268	4,168	2,720	2,054	Ful	C++	Qt	Non	Lin	My	Lin	Raw	Rea	0
cutelyst-thread-mysq	41,335	8,197	4,156	2,737	2,016	Ful	C++	Qt	Non	Lin	My	Lin	Raw	Rea	0
cutelyst-thread-post	58,176	11,345	5,639	3,752	2,801	Ful	C++	Qt	Non	Lin	Pg	Lin	Raw	Rea	0
cutelyst-thread-post	58,141	11,329	5,591	3,742	2,794	Ful	C++	Qt	Non	Lin	Pg	Lin	Raw	Rea	0
django	9,764	2,611	1,359	920	683	Ful	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
django-postgresql	10,873	2,759	1,449	979	710	Ful	Py	Non	Mei	Lin	Pg	Lin	Ful	Rea	0
django-py3	13,486	3,363	1,724	1,169	857	Ful	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
dropwizard-jdbi-post	31,669	6,781	3,492	2,340	1,749	Ful	Jav	JAX	Jty	Lin	Pg	Lin	Mcr	Rea	0
dropwizard-mongodb	22,921	4,395	2,189	1,170	868	Ful	Jav	JAX	Jty	Lin	Mo	Lin	Ful	Rea	17,055
dropwizard-mysql	46,292	13,471	5,970	3,935	2,820	Ful	Jav	JAX	Jty	Lin	My	Lin	Ful	Rea	0
dropwizard-postgres	47,243	14,347	6,713	4,360	3,243	Ful	Jav	JAX	Jty	Lin	Pg	Lin	Ful	Rea	0
falcore	28,396	7,347	3,627	2,379	1,650	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
ffead-cpp	0	0	0	0	0	Ful	C++	ffe	ffe	Lin	Mo	Lin	Ful	Rea	0
ffead-cpp-apache-mon	0	0	0	0	0	Ful	C++	ffe	Apc	Lin	Mo	Lin	Ful	Rea	0
ffead-cpp-apache-mys	0	0	0	0	0	Ful	C++	ffe	Apc	Lin	My	Lin	Ful	Rea	0
ffead-cpp-apache-pos	4,029	833	418	280	207	Ful	C++	ffe	Apc	Lin	Pg	Lin	Ful	Rea	0
ffead-cpp-mysql	0	0	0	0	0	Ful	C++	ffe	ffe	Lin	My	Lin	Ful	Rea	0
ffead-cpp-nginx-mong	0	0	0	0	0	Ful	C++	ffe	ngx	Lin	Mo	Lin	Ful	Rea	0
ffead-cpp-nginx-mysq	0	0	0	0	0	Ful	C++	ffe	ngx	Lin	My	Lin	Ful	Rea	0
ffead-cpp-nginx-post	4,005	832	417	277	207	Ful	C++	ffe	ngx	Lin	Pg	Lin	Ful	Rea	0
ffead-cpp-postgresql	0	0	0	0	0	Ful	C++	ffe	ffe	Lin	Pg	Lin	Ful	Rea	0
fintrospect	42,887	9,262	4,555	3,061	2,241	Mcr	Sca	Nty	Non	Lin	My	Lin	Raw	Rea	0
flask	10,196	3,387	1,895	1,465	1,077	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
flask-mysql-raw	25,558	7,389	3,706	2,494	1,846	Mcr	Py	Mei	Non	Lin	My	Lin	Raw	Rea	0
flask-nginx-uwsgi	5,724	3,482	1,941	1,274	870	Mcr	Py	Non	ngx	Lin	My	Lin	Ful	Rea	0
flask-py3	11,731	3,904	2,309	1,689	1,155	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
flask-pypy2	1,100	324	186	130	90	Mcr	Py	Non	Tor	Lin	My	Lin	Ful	Rea	0
flask-pypy2-mysql-ra	3,227	2,083	1,770	1,507	1,270	Mcr	Py	Tor	Non	Lin	My	Lin	Raw	Rea	0
gemini-mysql	17,155	3,191	1,587	1,121	819	Ful	Jav	Svt	Res	Lin	My	Lin	Mcr	Rea	0
gemini-postgres	19,641	4,135	1,943	1,381	1,102	Ful	Jav	Svt	Res	Lin	Pg	Lin	Mcr	Rea	0
gin	30,284	7,673	3,850	2,416	1,649	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
goji	23,627	6,672	3,066	1,958	1,395	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
http4s	41,170	11,322	4,259	671	116	Mcr	Sca	NIO	bla	Lin	Pg	Lin	Raw	Rea	35
jawn	61,622	12,890	6,458	4,318	3,252	Ful	Jav	Svt	Utw	Lin	Pg	Lin	Raw	Rea	0
jooby	43,912	9,021	4,539	3,022	2,264	Ful	Jav	Nty	Non	Lin	Pg	Lin	Raw	Rea	0
kami	29,253	5,392	2,589	1,652	1,185	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
klein	208	59	34	12	8	Mcr	Py	Non	Twi	Lin	My	Lin	Ful	Rea	40
minijax	21,457	7,037	4,026	2,838	2,182	Ful	Jav	JAX	Jty	Lin	My	Lin	Ful	Rea	0
morepath	21,254	6,556	3,521	2,389	1,789	Mcr	Py	Mei	Gun	Lin	Pg	Lin	Ful	Rea	0
ninja-standalone	7,490	2,563	1,316	872	653	Ful	Jav	Jty	Non	Lin	My	Lin	Ful	Rea	0
cherrypy	167	67	35	20	8	Mcr	Py	Non	Non	Lin	My	Lin	Ful	Rea	330
cherrypy-py3	221	79	39	22	15	Mcr	Py	Non	Non	Lin	My	Lin	Ful	Rea	192
cppcms	13,477	4,487	2,241	1,484	1,115	Plt	C++	Non	Non	Lin	My	Lin	Raw	Rea	0
cppcms-direct	10,112	4,521	2,249	1,483	1,109	Plt	C++	Non	Non	Lin	My	Lin	Raw	Rea	0
cppcms-postgres	14,190	6,346	3,195	2,130	1,589	Plt	C++	Non	Non	Lin	Pg	Lin	Raw	Rea	0
echo	15,717	4,817	2,705	1,851	1,381	Mcr	Go	Non	Non	Lin	Pg	Lin	Raw	Rea	0
fasthttp	11,990	5,369	3,289	2,392	1,840	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
fasthttp-postgresql	38,933	11,599	6,500	3,880	2,702	Plt	Go	Non	Non	Lin	Pg	Lin	Raw	Rea	0
go	15,048	4,989	2,733	1,908	1,422	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
go-interpolate	16,464	6,361	3,727	2,560	1,913	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
go-mongo	6,440	1,271	627	409	294	Plt	Go	Non	Non	Lin	Mo	Lin	Raw	Rea	0
go-postgres	16,683	5,037	2,693	1,853	1,412	Plt	Go	Non	Non	Lin	Pg	Lin	Raw	Rea	0
go-prefork	29,425	5,720	2,941	1,908	1,481	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
light-4j-postgresql	41,361	8,278	4,129	2,769	2,058	Plt	Jav	Lig	Non	Lin	Pg	Lin	Raw	Rea	0
revel-jet	23,070	4,676	2,422	1,641	1,272	Ful	Go	Non	Non	Lin	My	Lin	Mcr	Rea	0
revel-qbs	17,944	3,900	1,991	1,343	1,003	Ful	Go	Non	Non	Lin	My	Lin	Mcr	Rea	0
revel-raw	27,235	5,939	2,958	1,991	1,467	Ful	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
servlet-postgres-raw	28,788	5,862	2,958	1,974	1,471	Plt	Jav	Svt	Res	Lin	Pg	Lin	Raw	Rea	0
servlet-raw	36,111	7,669	3,705	2,494	1,850	Plt	Jav	Svt	Res	Lin	My	Lin	Raw	Rea	0
tornado	9,727	4,312	2,530	1,824	1,398	Plt	Py	Non	Tor	Lin	Mo	Lin	Raw	Rea	0
tornado-py3	10,745	3,333	1,834	1,240	940	Plt	Py	Non	Tor	Lin	Mo	Lin	Raw	Rea	0
tornado-pypy2	28,039	11,464	6,208	4,221	3,185	Plt	Py	Non	Tor	Lin	Mo	Lin	Raw	Rea	0
undertow-mongodb	29,446	9,341	4,818	3,165	2,404	Plt	Jav	Utw	Non	Lin	Mo	Lin	Raw	Rea	10,745
undertow-mongodb-asy	25,474	9,191	4,970	3,614	2,873	Plt	Jav	Utw	Non	Lin	Mo	Lin	Raw	Rea	18
undertow-mysql	48,765	9,911	4,873	3,274	2,453	Plt	Jav	Utw	Non	Lin	My	Lin	Raw	Rea	0
undertow-postgresql	66,883	13,290	6,694	4,451	3,352	Plt	Jav	Utw	Non	Lin	Pg	Lin	Raw	Rea	0
vertx-postgres	33,182	10,703	6,243	4,475	3,478	Plt	Jav	Ver	Non	Lin	Pg	Lin	Raw	Rea	0
wildfly-ee7	23,580	5,845	2,530	2,233	1,633	Ful	Jav	Svt	Wil	Lin	My	Lin	Ful	Rea	0
play2-java-ebean-hik	11,223	3,374	1,872	1,278	956	Ful	Jav	Akk	Non	Lin	My	Lin	Ful	Rea	10
play2-java-ebean-hik	11,012	3,404	1,852	1,268	966	Ful	Jav	Nty	Non	Lin	My	Lin	Ful	Rea	7
play2-java-jooq-hika	14,351	3,849	2,373	1,365	1,171	Ful	Jav	Akk	Non	Lin	My	Lin	Ful	Rea	0
play2-java-jooq-hika	14,623	4,162	2,425	1,376	1,214	Ful	Jav	Nty	Non	Lin	My	Lin	Ful	Rea	0
play2-java-jpa-hikar	11,187	3,615	2,027	1,452	1,110	Ful	Jav	Akk	Non	Lin	My	Lin	Ful	Rea	13
play2-java-jpa-hikar	11,552	3,770	2,137	1,420	1,087	Ful	Jav	Nty	Non	Lin	My	Lin	Ful	Rea	11
play2-scala-anorm-li	26,954	5,936	2,922	1,988	1,480	Ful	Sca	Akk	Non	Lin	My	Lin	Ful	Rea	0
play2-scala-anorm-li	26,321	5,675	2,850	1,954	1,474	Ful	Sca	Nty	Non	Lin	My	Lin	Ful	Rea	0
play2-scala-reactive	1,690	318	136	73	56	Ful	Sca	Akk	Non	Lin	Mo	Lin	Ful	Rea	0
play2-scala-reactive	1,719	309	147	76	48	Ful	Sca	Nty	Non	Lin	Mo	Lin	Ful	Rea	0
play2-scala-slick	8,016	1,567	765	496	360	Ful	Sca	Akk	Non	Lin	My	Lin	Ful	Rea	99
play2-scala-slick-ne	7,673	1,522	742	480	354	Ful	Sca	Nty	Non	Lin	My	Lin	Ful	Rea	0
pyramid	12,326	4,516	2,509	1,798	1,407	Ful	Py	Non	Mei	Lin	Pg	Lin	Ful	Rea	0
pyramid-py2	10,527	3,922	2,236	1,557	1,209	Ful	Py	Non	Mei	Lin	Pg	Lin	Ful	Rea	0
revenj-jvm	23,999	8,890	4,694	3,067	2,255	Ful	Jav	Svt	Res	Lin	Pg	Lin	Ful	Rea	0
silicon	25,260	7,421	3,907	2,637	2,038	Mcr	C++	Non	mic	Lin	My	Lin	Ful	Rea	0
spark	1,560	472	259	129	85	Mcr	Jav	Svt	Res	Lin	My	Lin	Ful	Rea	0
spring	13,032	2,596	1,301	857	659	Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	0
treefrog	14,236	2,873	1,430	936	703	Ful	C++	Non	Non	Lin	My	Lin	Mcr	Rea	0
treefrog-hybrid	18,916	3,795	1,889	1,239	927	Ful	C++	Non	Non	Lin	My	Lin	Mcr	Rea	1
treefrog-mongodb	52,877	10,599	5,245	3,487	2,608	Ful	C++	Non	Non	Lin	Mo	Lin	Mcr	Rea	0
treefrog-postgres	12,267	2,381	1,219	793	568	Ful	C++	Non	Non	Lin	Pg	Lin	Mcr	Rea	0
turbogears	10,154	3,036	1,626	1,249	950	Mcr	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
ulib-mongodb	66,348	13,259	6,653	4,412	3,313	Plt	C++	Non	ULi	Lin	Mo	Lin	Mcr	Rea	0
ulib-mysql	49,331	9,802	4,872	3,295	2,468	Plt	C++	Non	ULi	Lin	My	Lin	Mcr	Rea	0
ulib-postgres	25,306	11,141	6,740	4,809	3,729	Plt	C++	Non	ULi	Lin	Pg	Lin	Mcr	Rea	0
ulib-postgres_fit	31,322	14,906	9,048	6,363	4,886	Plt	C++	Non	ULi	Lin	Pg	Lin	Mcr	Rea	0
uvicorn	9,714	2,838	1,494	1,028	760	Plt	Py	Non	Non	Lin	Pg	Lin	Raw	Rea	0
vertx-web-mongodb	29,188	5,795	2,872	1,922	1,433	Mcr	Jav	vtx	Non	Lin	Mo	Lin	Raw	Rea	23
vertx-web-postgres	46,859	14,113	7,662	5,221	4,002	Mcr	Jav	vtx	Non	Lin	Pg	Lin	Raw	Rea	0
vertx-web-susom-post	36,978	11,610	2,987	837	828	Mcr	Jav	vtx	Non	Lin	Pg	Lin	Raw	Rea	0
web2py	7,003	3,924	1,114	267	107	Ful	Py	Mei	Non	Lin	My	Lin	Ful	Rea	1,055
web2py-optimized	13,010	3,201	684	543	513	Ful	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
webware	0	0	0	0	0	Mcr	Py	Non	Non	Lin	My	Lin	Ful	Rea	795
weppy	17,772	4,904	2,685	1,956	1,352	Ful	Py	Mei	Non	Lin	Pg	Lin	Ful	Rea	0
weppy-nginx-uwsgi	4,699	6,183	3,098	1,834	1,181	Ful	Py	uWS	ngx	Lin	Pg	Lin	Ful	Rea	0
weppy-py3	22,552	6,289	3,175	2,140	1,483	Ful	Py	Mei	Non	Lin	Pg	Lin	Ful	Rea	0
weppy-pypy2	2,971	1,032	407	19	11	Ful	Py	Tor	Non	Lin	Pg	Lin	Ful	Rea	142
wheezyweb	12,269	3,106	1,561	1,086	777	Mcr	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
wheezyweb-py3	— Did not complete —						Mcr	Py	Non	Mei	Lin	My	Lin	Ful	Rea	—
wicket	19,105	4,474	2,290	1,566	1,165	Ful	Jav	Svt	Res	Lin	My	Lin	Ful	Rea	0
wt	41,512	8,246	4,070	2,741	2,040	Ful	C++	Non	Non	Lin	My	Lin	Ful	Rea	45
wt-postgres	55,899	11,980	5,972	3,981	2,975	Ful	C++	Non	Non	Lin	Pg	Lin	Ful	Rea	6

Latency of 20-update responses, Dell servers at ServerCentral

Framework	Average latency (lower is better)		σ (SD)	Max	Errors
minijax	89.3 ms	1.4%	38.6 ms	876.6 ms	0
ulib-postgres_fit	104.7 ms	1.6%	44.1 ms	287.9 ms	0
vertx-web-postgres	125.2 ms	1.9%	35.2 ms	365.5 ms	0
ulib-postgres	143.0 ms	2.2%	94.1 ms	946.6 ms	0
undertow-postgresql	149.6 ms	2.3%	67.2 ms	713.0 ms	0
vertx-postgres	150.6 ms	2.3%	82.5 ms	1200.0 ms	0
jawn	153.2 ms	2.4%	66.7 ms	745.4 ms	0
ulib-mongodb	154.3 ms	2.4%	75.7 ms	446.1 ms	0
dropwizard-postgres	157.3 ms	2.4%	47.1 ms	489.2 ms	0
cutelyst-nginx-postg	165.8 ms	2.5%	9.7 ms	196.1 ms	0
tornado-pypy2	166.9 ms	2.6%	129.1 ms	1170.0 ms	0
cutelyst-pf-postgres	168.5 ms	2.6%	74.5 ms	661.3 ms	0
undertow-mongodb-asy	178.7 ms	2.7%	164.8 ms	887.6 ms	18
cutelyst-thread-post	183.7 ms	2.8%	91.5 ms	647.9 ms	0
cutelyst-thread-post	184.4 ms	2.8%	82.9 ms	731.6 ms	0
fasthttp-postgresql	186.1 ms	2.9%	72.0 ms	636.0 ms	0
dropwizard-mysql	189.8 ms	2.9%	124.2 ms	1070.0 ms	0
undertow-mongodb	191.6 ms	2.9%	157.6 ms	818.0 ms	10,745
treefrog-mongodb	192.0 ms	3.0%	111.8 ms	1170.0 ms	0
ulib-mysql	206.4 ms	3.2%	91.3 ms	571.3 ms	0
undertow-mysql	208.9 ms	3.2%	87.4 ms	1010.0 ms	0
fintrospect	211.4 ms	3.3%	68.1 ms	1010.0 ms	0
actframework-morphia	214.1 ms	3.3%	42.5 ms	433.3 ms	0
api_hour-mysql	219.0 ms	3.4%	253.0 ms	1130.0 ms	188
jooby	224.6 ms	3.5%	45.7 ms	583.0 ms	0
silicon	233.0 ms	3.6%	132.7 ms	1850.0 ms	0
akka-http	233.7 ms	3.6%	47.7 ms	575.0 ms	0
dropwizard-mongodb	244.6 ms	3.8%	177.9 ms	539.8 ms	17,055
light-4j-postgresql	246.7 ms	3.8%	31.0 ms	414.2 ms	0
cutelyst-thread-mysq	248.6 ms	3.8%	116.1 ms	835.5 ms	0
cpoll_cppsp-raw	251.0 ms	3.9%	69.5 ms	938.2 ms	0
cutelyst-thread-mysq	254.1 ms	3.9%	110.1 ms	907.8 ms	0
go-interpolate	255.3 ms	3.9%	67.7 ms	710.6 ms	0
servlet-raw	259.3 ms	4.0%	90.0 ms	769.7 ms	0
flask-mysql-raw	264.8 ms	4.1%	108.1 ms	759.4 ms	0
fasthttp	266.0 ms	4.1%	85.0 ms	884.6 ms	0
bottle-mysql-raw	274.4 ms	4.2%	87.2 ms	992.5 ms	0
dropwizard-jdbi-post	276.2 ms	4.2%	59.5 ms	552.3 ms	0
cutelyst-nginx-mysql	276.9 ms	4.3%	33.8 ms	404.0 ms	0
cutelyst-pf-mysql	279.2 ms	4.3%	123.3 ms	1060.0 ms	0
vertx-web-susom-post	279.3 ms	4.3%	561.6 ms	3090.0 ms	0
actframework-ebean-m	286.1 ms	4.4%	33.7 ms	453.8 ms	0
actframework-hiberna	294.4 ms	4.5%	34.5 ms	555.2 ms	0
falcore	295.6 ms	4.5%	92.1 ms	809.7 ms	0
gin	295.6 ms	4.5%	91.1 ms	928.9 ms	0
actframework-ebean-p	296.4 ms	4.6%	49.6 ms	570.6 ms	0
actframework-hiberna	301.1 ms	4.6%	48.5 ms	543.5 ms	0
morepath	309.2 ms	4.8%	265.6 ms	3100.0 ms	0
cppcms-postgres	318.5 ms	4.9%	72.7 ms	707.1 ms	0
wt-postgres	321.6 ms	4.9%	869.3 ms	8000.0 ms	6
go-prefork	324.4 ms	5.0%	139.8 ms	1040.0 ms	0
cutelyst-pf-mysql-ep	329.4 ms	5.1%	310.5 ms	3390.0 ms	0
revel-raw	332.5 ms	5.1%	98.8 ms	790.6 ms	0
servlet-postgres-raw	332.7 ms	5.1%	98.6 ms	794.1 ms	0
play2-scala-anorm-li	333.4 ms	5.1%	55.5 ms	669.4 ms	0
wt	337.9 ms	5.2%	914.5 ms	7980.0 ms	45
go-postgres	339.0 ms	5.2%	84.5 ms	700.1 ms	0
play2-scala-anorm-li	342.2 ms	5.3%	40.1 ms	653.3 ms	0
pyramid	348.8 ms	5.4%	170.9 ms	1250.0 ms	0
goji	349.1 ms	5.4%	96.8 ms	885.2 ms	0
tornado	350.3 ms	5.4%	117.4 ms	961.4 ms	0
vertx-web-mongodb	352.4 ms	5.4%	126.1 ms	1050.0 ms	23
go	355.8 ms	5.5%	41.2 ms	511.0 ms	0
aah	359.5 ms	5.5%	64.2 ms	785.2 ms	0
revenj-jvm	366.0 ms	5.6%	731.4 ms	7980.0 ms	0
echo	366.4 ms	5.6%	43.1 ms	574.9 ms	0
weppy-py3	378.9 ms	5.8%	355.3 ms	5220.0 ms	0
revel-jet	396.9 ms	6.1%	97.2 ms	995.1 ms	0
pyramid-py2	398.9 ms	6.1%	183.3 ms	1490.0 ms	0
flask-pypy2-mysql-ra	400.9 ms	6.2%	316.0 ms	5800.0 ms	0
weppy	409.9 ms	6.3%	429.5 ms	4890.0 ms	0
play2-java-jooq-hika	410.5 ms	6.3%	30.0 ms	596.7 ms	0
wicket	417.0 ms	6.4%	134.7 ms	1370.0 ms	0
play2-java-jooq-hika	424.6 ms	6.5%	60.0 ms	1150.0 ms	0
kami	425.5 ms	6.5%	90.2 ms	1070.0 ms	0
weppy-nginx-uwsgi	426.5 ms	6.6%	66.8 ms	924.1 ms	0
activeweb-jackson	436.7 ms	6.7%	96.6 ms	906.2 ms	0
play2-java-jpa-hikar	445.2 ms	6.8%	71.8 ms	1150.0 ms	13
cppcms	452.4 ms	7.0%	55.2 ms	791.9 ms	0
cppcms-direct	454.6 ms	7.0%	55.3 ms	810.2 ms	0
play2-java-jpa-hikar	462.6 ms	7.1%	54.5 ms	879.6 ms	11
api_hour	465.7 ms	7.2%	427.7 ms	4040.0 ms	0
flask-py3	466.7 ms	7.2%	392.2 ms	3280.0 ms	0
cherrypy	478.5 ms	7.4%	67.6 ms	725.9 ms	330
activeweb	482.8 ms	7.4%	114.0 ms	988.0 ms	0
revel-qbs	485.8 ms	7.5%	94.9 ms	864.3 ms	0
flask	496.4 ms	7.6%	423.7 ms	4010.0 ms	0
aiohttp-pg-raw	498.4 ms	7.7%	157.8 ms	1300.0 ms	0
play2-java-ebean-hik	503.9 ms	7.8%	57.6 ms	714.2 ms	7
tornado-py3	507.9 ms	7.8%	123.1 ms	1050.0 ms	0
play2-java-ebean-hik	514.1 ms	7.9%	81.2 ms	1280.0 ms	10
treefrog-hybrid	529.1 ms	8.1%	91.3 ms	1100.0 ms	1
turbogears	539.4 ms	8.3%	437.6 ms	3720.0 ms	0
aiohttp	540.2 ms	8.3%	166.3 ms	1220.0 ms	0
django-py3	546.0 ms	8.4%	184.8 ms	1290.0 ms	0
bottle-py3	546.3 ms	8.4%	489.7 ms	4200.0 ms	0
flask-nginx-uwsgi	576.0 ms	8.9%	92.4 ms	1090.0 ms	0
bottle	589.8 ms	9.1%	535.2 ms	6030.0 ms	0
beego-orm-mysql	621.6 ms	9.6%	120.5 ms	1310.0 ms	0
cherrypy-py3	624.0 ms	9.6%	829.1 ms	4760.0 ms	192
uvicorn	633.3 ms	9.7%	200.6 ms	1560.0 ms	0
gemini-postgres	645.0 ms	9.9%	819.1 ms	6990.0 ms	0
gemini-mysql	649.1 ms	10.0%	530.8 ms	4180.0 ms	0
wheezyweb	649.5 ms	10.0%	575.3 ms	4680.0 ms	0
bottle-nginx-uwsgi	676.9 ms	10.4%	109.8 ms	1390.0 ms	0
django-postgresql	677.1 ms	10.4%	221.9 ms	1470.0 ms	0
treefrog	682.2 ms	10.5%	162.6 ms	1930.0 ms	0
django	698.6 ms	10.7%	268.3 ms	1750.0 ms	0
ninja-standalone	737.8 ms	11.4%	204.9 ms	2770.0 ms	0
spring	757.0 ms	11.6%	99.0 ms	1060.0 ms	0
treefrog-postgres	829.6 ms	12.8%	264.5 ms	2590.0 ms	0
wildfly-ee7	1010.0 ms	15.5%	1530.0 ms	7900.0 ms	0
web2py-optimized	1100.0 ms	16.9%	1370.0 ms	8000.0 ms	0
play2-scala-slick	1200.0 ms	18.5%	1080.0 ms	7980.0 ms	99
play2-scala-slick-ne	1390.0 ms	21.4%	906.1 ms	5390.0 ms	0
go-mongo	1520.0 ms	23.4%	149.9 ms	1770.0 ms	0
ffead-cpp-apache-pos	2170.0 ms	33.4%	569.7 ms	2610.0 ms	0
ffead-cpp-nginx-post	2260.0 ms	34.8%	474.6 ms	2640.0 ms	0
web2py	2390.0 ms	36.8%	1580.0 ms	7950.0 ms	1,055
bottle-pypy2	2820.0 ms	43.4%	1970.0 ms	7980.0 ms	0
flask-pypy2	3290.0 ms	50.6%	2050.0 ms	7980.0 ms	0
http4s	3350.0 ms	51.5%	2330.0 ms	8000.0 ms	35
weppy-pypy2	3480.0 ms	53.5%	2550.0 ms	7960.0 ms	142
spark	4050.0 ms	62.3%	1570.0 ms	7760.0 ms	0
play2-scala-reactive	5870.0 ms	90.3%	1970.0 ms	7980.0 ms	0
klein	5960.0 ms	91.7%	1480.0 ms	7780.0 ms	40
play2-scala-reactive	6350.0 ms	97.7%	1270.0 ms	7990.0 ms	0
webware	6500.0 ms	100.0%	668.3 ms	8000.0 ms	795

Framework overhead of 20-update responses, Dell servers at ServerCentral

Framework best		vs	Platform best		Ratio (higher is better)
treefrog-mongodb	2,608	vs	treefrog	703	371.0%
tornado-pypy2	3,185	vs	tornado	1,398	227.8%
dropwizard-postgres	3,243	vs	servlet-postgres-raw	1,471	220.5%
fasthttp-postgresql	2,702	vs	go	1,422	190.0%
dropwizard-mysql	2,820	vs	servlet-raw	1,850	152.4%
wt-postgres	2,975	vs	wt	2,040	145.8%
go-interpolate	1,913	vs	go	1,422	134.5%
treefrog-hybrid	927	vs	treefrog	703	131.9%
fasthttp	1,840	vs	go	1,422	129.4%
dropwizard-jdbi-post	1,749	vs	servlet-postgres-raw	1,471	118.9%
falcore	1,650	vs	go	1,422	116.0%
gin	1,649	vs	go	1,422	116.0%
go-prefork	1,481	vs	go	1,422	104.1%
revel-raw	1,467	vs	go	1,422	103.2%
go-postgres	1,412	vs	go	1,422	99.3%
aah	1,406	vs	go	1,422	98.9%
goji	1,395	vs	go	1,422	98.1%
echo	1,381	vs	go	1,422	97.1%
revel-jet	1,272	vs	go	1,422	89.5%
kami	1,185	vs	go	1,422	83.3%
servlet-postgres-raw	1,471	vs	servlet-raw	1,850	79.5%
revel-qbs	1,003	vs	go	1,422	70.5%
play2-scala-anorm-li	1,480	vs	akka-http	2,117	69.9%
tornado-py3	940	vs	tornado	1,398	67.2%
beego-orm-mysql	801	vs	go	1,422	56.3%
play2-java-jooq-hika	1,171	vs	akka-http	2,117	55.3%
play2-java-jpa-hikar	1,110	vs	akka-http	2,117	52.4%
play2-java-ebean-hik	956	vs	akka-http	2,117	45.2%
go-mongo	294	vs	go	1,422	20.7%
play2-scala-slick	360	vs	akka-http	2,117	17.0%
play2-scala-reactive	56	vs	akka-http	2,117	2.6%

20-updates (bar)

Data table

Latency

Framework overhead

20-updates (bar)

Data table

Latency

Framework overhead

Requirements summary

This test exercises database writes. Each request is processed by fetching multiple rows from a simple database table, converting the rows to in-memory objects, modifying one attribute of each object in memory, updating each associated row in the database individually, and then serializing the list of objects as a JSON response. The test is run multiple times: testing 1, 5, 10, 15, and 20 updates per request. Note that the number of statements per request is twice the number of updates since each update is paired with one query to fetch the object. All tests are run at 256 concurrency.

The response is analogous to the multiple-query test. Example response for 10 updates:

For a more detailed description of the requirements, see the Source Code and Requirements section.

Best fortunes responses per second, Dell servers at ServerCentral(151 tests)

Framework	Best performance (higher is better)		Cls	Lng	Plt	FE	Aos	DB	Dos	Orm	IA	Errors
ulib-postgres_fit	189,705	100.0%	Plt	C++	Non	ULi	Lin	Pg	Lin	Mcr	Rea	0
ulib-postgres	184,679	97.4%	Plt	C++	Non	ULi	Lin	Pg	Lin	Mcr	Rea	0
ulib-mongodb	174,785	92.1%	Plt	C++	Non	ULi	Lin	Mo	Lin	Mcr	Rea	0
gemini-postgres	173,928	91.7%	Ful	Jav	Svt	Res	Lin	Pg	Lin	Mcr	Rea	0
cutelyst-thread-post	167,932	88.5%	Ful	C++	Qt	Non	Lin	Pg	Lin	Raw	Rea	0
light-4j-postgresql	167,583	88.3%	Plt	Jav	Lig	Non	Lin	Pg	Lin	Raw	Rea	0
cutelyst-thread-post	167,095	88.1%	Ful	C++	Qt	Non	Lin	Pg	Lin	Raw	Rea	0
proteus-mysql	162,613	85.7%	Mcr	Jav	Utw	Non	Lin	My	Lin	Raw	Rea	0
vertx-postgres	146,000	77.0%	Plt	Jav	Ver	Non	Lin	Pg	Lin	Raw	Rea	0
proteus	139,484	73.5%	Mcr	Jav	Utw	Non	Lin	Pg	Lin	Raw	Rea	0
ulib-mysql	136,976	72.2%	Plt	C++	Non	ULi	Lin	My	Lin	Mcr	Rea	0
undertow-postgresql	134,155	70.7%	Plt	Jav	Utw	Non	Lin	Pg	Lin	Raw	Rea	0
fasthttp-postgresql	124,514	65.6%	Plt	Go	Non	Non	Lin	Pg	Lin	Raw	Rea	0
rapidoid-mysql	117,301	61.8%	Plt	Jav	Rap	Non	Lin	My	Lin	Mcr	Rea	0
treefrog-mongodb	115,114	60.7%	Ful	C++	Non	Non	Lin	Mo	Lin	Mcr	Rea	0
cpoll_cppsp-raw	112,982	59.6%	Plt	C++	Non	Non	Lin	My	Lin	Raw	Rea	0
go-prefork	112,122	59.1%	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
rapidoid-postgres	111,093	58.6%	Plt	Jav	Rap	Non	Lin	Pg	Lin	Mcr	Rea	0
cutelyst-pf-postgres	110,129	58.1%	Ful	C++	Qt	Non	Lin	Pg	Lin	Raw	Rea	0
undertow-mysql	108,367	57.1%	Plt	Jav	Utw	Non	Lin	My	Lin	Raw	Rea	0
gemini-mysql	108,138	57.0%	Ful	Jav	Svt	Res	Lin	My	Lin	Mcr	Rea	0
cutelyst-nginx-postg	100,941	53.2%	Ful	C++	Qt	ngx	Lin	Pg	Lin	Raw	Rea	0
cutelyst-thread-mysq	98,720	52.0%	Ful	C++	Qt	Non	Lin	My	Lin	Raw	Rea	0
cutelyst-thread-mysq	98,159	51.7%	Ful	C++	Qt	Non	Lin	My	Lin	Raw	Rea	0
undertow-mongodb	94,833	50.0%	Plt	Jav	Utw	Non	Lin	Mo	Lin	Raw	Rea	0
revenj-jvm	87,123	45.9%	Ful	Jav	Svt	Res	Lin	Pg	Lin	Ful	Rea	0
fintrospect	85,893	45.3%	Mcr	Sca	Nty	Non	Lin	My	Lin	Raw	Rea	0
actframework-ebean-p	84,508	44.5%	Ful	Jav	Utw	Non	Lin	Pg	Lin	Ful	Rea	0
actframework-hiberna	77,964	41.1%	Ful	Jav	Utw	Non	Lin	Pg	Lin	Ful	Rea	0
cutelyst-pf-mysql	76,323	40.2%	Ful	C++	Qt	Non	Lin	My	Lin	Raw	Rea	0
cutelyst-nginx-mysql	73,206	38.6%	Ful	C++	Qt	ngx	Lin	My	Lin	Raw	Rea	0
jooby	72,687	38.3%	Ful	Jav	Nty	Non	Lin	Pg	Lin	Raw	Rea	0
silicon	69,547	36.7%	Mcr	C++	Non	mic	Lin	My	Lin	Ful	Rea	0
fasthttp	65,152	34.3%	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
actframework-ebean-m	64,286	33.9%	Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	0
grizzly-jersey	62,635	33.0%	Mcr	Jav	JAX	Non	Lin	My	Lin	Ful	Rea	0
cutelyst-pf-mysql-ep	60,412	31.8%	Ful	C++	Qt	Non	Lin	My	Lin	Raw	Rea	0
kami	59,441	31.3%	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
treefrog-postgres	56,352	29.7%	Ful	C++	Non	Non	Lin	Pg	Lin	Mcr	Rea	0
actframework-hiberna	56,241	29.6%	Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	0
treefrog	55,606	29.3%	Ful	C++	Non	Non	Lin	My	Lin	Mcr	Rea	0
http4s	53,768	28.3%	Mcr	Sca	NIO	bla	Lin	Pg	Lin	Raw	Rea	0
goji	53,466	28.2%	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
go-interpolate	51,216	27.0%	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
go-postgres	50,347	26.5%	Plt	Go	Non	Non	Lin	Pg	Lin	Raw	Rea	0
go	49,440	26.1%	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
servlet-postgres-raw	47,985	25.3%	Plt	Jav	Svt	Res	Lin	Pg	Lin	Raw	Rea	0
servlet-raw	47,862	25.2%	Plt	Jav	Svt	Res	Lin	My	Lin	Raw	Rea	0
actframework-hiberna	47,433	25.0%	Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	0
vertx-web-postgres	47,214	24.9%	Mcr	Jav	vtx	Non	Lin	Pg	Lin	Raw	Rea	0
actframework-hiberna	47,067	24.8%	Ful	Jav	Utw	Non	Lin	Pg	Lin	Ful	Rea	0
treefrog-hybrid	46,316	24.4%	Ful	C++	Non	Non	Lin	My	Lin	Mcr	Rea	8
bottle-mysql-raw	45,787	24.1%	Mcr	Py	Mei	Non	Lin	My	Lin	Raw	Rea	0
undertow-mongodb-asy	45,602	24.0%	Plt	Jav	Utw	Non	Lin	Mo	Lin	Raw	Rea	0
vertx-web-mongodb	44,949	23.7%	Mcr	Jav	vtx	Non	Lin	Mo	Lin	Raw	Rea	0
revel-raw	44,742	23.6%	Ful	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
gin	44,347	23.4%	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
revel-jet	42,558	22.4%	Ful	Go	Non	Non	Lin	My	Lin	Mcr	Rea	0
tornado-pypy2	40,210	21.2%	Plt	Py	Non	Tor	Lin	Mo	Lin	Raw	Rea	0
vertx-web-susom-post	39,877	21.0%	Mcr	Jav	vtx	Non	Lin	Pg	Lin	Raw	Rea	0
revel-qbs	38,816	20.5%	Ful	Go	Non	Non	Lin	My	Lin	Mcr	Rea	0
minijax	38,574	20.3%	Ful	Jav	JAX	Jty	Lin	My	Lin	Ful	Rea	0
falcore	38,055	20.1%	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
echo	36,725	19.4%	Mcr	Go	Non	Non	Lin	Pg	Lin	Raw	Rea	0
dropwizard-mysql	35,510	18.7%	Ful	Jav	JAX	Jty	Lin	My	Lin	Ful	Rea	0
dropwizard-postgres	33,729	17.8%	Ful	Jav	JAX	Jty	Lin	Pg	Lin	Ful	Rea	0
uvicorn	32,991	17.4%	Plt	Py	Non	Non	Lin	Pg	Lin	Raw	Rea	0
dropwizard-jdbi-post	31,744	16.7%	Ful	Jav	JAX	Jty	Lin	Pg	Lin	Mcr	Rea	0
aiohttp	31,528	16.6%	Mcr	Py	asy	Gun	Lin	Pg	Lin	Ful	Rea	0
aiohttp-pg-raw	30,971	16.3%	Mcr	Py	asy	Gun	Lin	Pg	Lin	Raw	Rea	0
wt	30,869	16.3%	Ful	C++	Non	Non	Lin	My	Lin	Ful	Rea	0
aah	30,534	16.1%	Ful	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
api_hour	30,337	16.0%	Mcr	Py	asy	Gun	Lin	Pg	Lin	Raw	Rea	0
wt-postgres	30,045	15.8%	Ful	C++	Non	Non	Lin	Pg	Lin	Ful	Rea	0
tapestry	29,473	15.5%	Ful	Jav	Svt	Res	Lin	My	Lin	Ful	Rea	0
jawn	29,316	15.5%	Ful	Jav	Svt	Utw	Lin	Pg	Lin	Raw	Rea	0
wheezyweb	27,862	14.7%	Mcr	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
play2-java-jpa-hikar	26,798	14.1%	Ful	Jav	Nty	Non	Lin	My	Lin	Ful	Rea	0
flask-mysql-raw	26,198	13.8%	Mcr	Py	Mei	Non	Lin	My	Lin	Raw	Rea	0
api_hour-mysql	26,068	13.7%	Mcr	Py	asy	Gun	Lin	My	Lin	Raw	Rea	0
play2-java-jpa-hikar	24,981	13.2%	Ful	Jav	Akk	Non	Lin	My	Lin	Ful	Rea	0
actframework-morphia	24,701	13.0%	Ful	Jav	Utw	Non	Lin	Mo	Lin	Ful	Rea	0
morepath	23,011	12.1%	Mcr	Py	Mei	Gun	Lin	Pg	Lin	Ful	Rea	0
spring	22,439	11.8%	Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	0
play2-java-jooq-hika	22,165	11.7%	Ful	Jav	Nty	Non	Lin	My	Lin	Ful	Rea	0
weppy-py3	21,806	11.5%	Ful	Py	Mei	Non	Lin	Pg	Lin	Ful	Rea	0
actframework-morphia	21,738	11.5%	Ful	Jav	Utw	Non	Lin	Mo	Lin	Ful	Rea	0
play2-scala-slick-ne	21,378	11.3%	Ful	Sca	Nty	Non	Lin	My	Lin	Ful	Rea	0
wicket	21,168	11.2%	Ful	Jav	Svt	Res	Lin	My	Lin	Ful	Rea	0
pyramid-py2	21,138	11.1%	Ful	Py	Non	Mei	Lin	Pg	Lin	Ful	Rea	0
play2-scala-reactive	20,649	10.9%	Ful	Sca	Nty	Non	Lin	Mo	Lin	Ful	Rea	0
pyramid	20,603	10.9%	Ful	Py	Non	Mei	Lin	Pg	Lin	Ful	Rea	0
weppy	20,183	10.6%	Ful	Py	Mei	Non	Lin	Pg	Lin	Ful	Rea	0
play2-scala-slick	19,990	10.5%	Ful	Sca	Akk	Non	Lin	My	Lin	Ful	Rea	0
undertow-jersey-hika	19,035	10.0%	Plt	Jav	JAX	Non	Lin	My	Lin	Ful	Rea	0
play2-java-jooq-hika	18,968	10.0%	Ful	Jav	Akk	Non	Lin	My	Lin	Ful	Rea	0
django-py3	17,814	9.4%	Ful	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
undertow-jersey-c3p0	17,594	9.3%	Plt	Jav	JAX	Non	Lin	My	Lin	Ful	Rea	0
actframework-ebean-p	17,301	9.1%	Ful	Jav	Utw	Non	Lin	Pg	Lin	Ful	Rea	0
bottle-py3	16,990	9.0%	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
play2-scala-anorm-li	16,898	8.9%	Ful	Sca	Nty	Non	Lin	My	Lin	Ful	Rea	0
cppcms-postgres	16,308	8.6%	Plt	C++	Non	Non	Lin	Pg	Lin	Raw	Rea	0
wildfly-ee7	15,657	8.3%	Ful	Jav	Svt	Wil	Lin	My	Lin	Ful	Rea	0
bottle	15,434	8.1%	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
play2-java-ebean-hik	15,376	8.1%	Ful	Jav	Akk	Non	Lin	My	Lin	Ful	Rea	0
cppcms	15,091	8.0%	Plt	C++	Non	Non	Lin	My	Lin	Raw	Rea	0
play2-scala-anorm-li	14,843	7.8%	Ful	Sca	Akk	Non	Lin	My	Lin	Ful	Rea	0
play2-java-ebean-hik	14,485	7.6%	Ful	Jav	Nty	Non	Lin	My	Lin	Ful	Rea	0
flask-py3	14,469	7.6%	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
turbogears	14,157	7.5%	Mcr	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
activeweb	13,986	7.4%	Ful	Jav	Svt	Non	Lin	My	Lin	Mcr	Rea	0
tornado-py3	13,277	7.0%	Plt	Py	Non	Tor	Lin	Mo	Lin	Raw	Rea	0
play2-scala-reactive	12,740	6.7%	Ful	Sca	Akk	Non	Lin	Mo	Lin	Ful	Rea	0
django-postgresql	12,673	6.7%	Ful	Py	Non	Mei	Lin	Pg	Lin	Ful	Rea	0
tornado	12,672	6.7%	Plt	Py	Non	Tor	Lin	Mo	Lin	Raw	Rea	0
flask	12,580	6.6%	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
dropwizard-mongodb	12,339	6.5%	Ful	Jav	JAX	Jty	Lin	Mo	Lin	Ful	Rea	0
django	11,643	6.1%	Ful	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
web2py-optimized	11,262	5.9%	Ful	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
go-mongo	10,864	5.7%	Plt	Go	Non	Non	Lin	Mo	Lin	Raw	Rea	0
cppcms-direct	10,857	5.7%	Plt	C++	Non	Non	Lin	My	Lin	Raw	Rea	0
flask-pypy2-mysql-ra	8,741	4.6%	Mcr	Py	Tor	Non	Lin	My	Lin	Raw	Rea	0
ninja-standalone	7,011	3.7%	Ful	Jav	Jty	Non	Lin	My	Lin	Ful	Rea	0
bottle-pypy2	6,912	3.6%	Mcr	Py	Tor	Non	Lin	My	Lin	Ful	Rea	0
akka-http	6,753	3.6%	Mcr	Sca	Akk	Non	Lin	My	Lin	Raw	Rea	0
web2py	6,326	3.3%	Ful	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
flask-nginx-uwsgi	5,658	3.0%	Mcr	Py	Non	ngx	Lin	My	Lin	Ful	Rea	0
weppy-nginx-uwsgi	5,123	2.7%	Ful	Py	uWS	ngx	Lin	Pg	Lin	Ful	Rea	0
bottle-nginx-uwsgi	5,015	2.6%	Mcr	Py	uWS	ngx	Lin	My	Lin	Ful	Rea	0
flask-pypy2	3,264	1.7%	Mcr	Py	Non	Tor	Lin	My	Lin	Ful	Rea	0
weppy-pypy2	3,132	1.7%	Ful	Py	Tor	Non	Lin	Pg	Lin	Ful	Rea	0
ffead-cpp-apache-pos	2,861	1.5%	Ful	C++	ffe	Apc	Lin	Pg	Lin	Ful	Rea	0
ffead-cpp-nginx-post	2,847	1.5%	Ful	C++	ffe	ngx	Lin	Pg	Lin	Ful	Rea	0
actframework-ebean-m	2,365	1.2%	Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	0
klein	266	0.1%	Mcr	Py	Non	Twi	Lin	My	Lin	Ful	Rea	0
cherrypy-py3	255	0.1%	Mcr	Py	Non	Non	Lin	My	Lin	Ful	Rea	0
spark	224	0.1%	Mcr	Jav	Svt	Res	Lin	My	Lin	Ful	Rea	0
cherrypy	197	0.1%	Mcr	Py	Non	Non	Lin	My	Lin	Ful	Rea	0
ffead-cpp	0	0.0%	Ful	C++	ffe	ffe	Lin	Mo	Lin	Ful	Rea	0
ffead-cpp-apache-mon	0	0.0%	Ful	C++	ffe	Apc	Lin	Mo	Lin	Ful	Rea	0
ffead-cpp-apache-mys	0	0.0%	Ful	C++	ffe	Apc	Lin	My	Lin	Ful	Rea	0
ffead-cpp-mysql	0	0.0%	Ful	C++	ffe	ffe	Lin	My	Lin	Ful	Rea	0
ffead-cpp-nginx-mong	0	0.0%	Ful	C++	ffe	ngx	Lin	Mo	Lin	Ful	Rea	0
ffead-cpp-nginx-mysq	0	0.0%	Ful	C++	ffe	ngx	Lin	My	Lin	Ful	Rea	0
ffead-cpp-postgresql	0	0.0%	Ful	C++	ffe	ffe	Lin	Pg	Lin	Ful	Rea	0
webware	0	0.0%	Mcr	Py	Non	Non	Lin	My	Lin	Ful	Rea	1,154
actframework-eclipse	—	Did not complete	Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	—
actframework-eclipse	—	Did not complete	Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	—
actframework-eclipse	—	Did not complete	Ful	Jav	Utw	Non	Lin	Pg	Lin	Ful	Rea	—
actframework-eclipse	—	Did not complete	Ful	Jav	Utw	Non	Lin	Pg	Lin	Ful	Rea	—
wheezyweb-py3	—	Did not complete	Mcr	Py	Non	Mei	Lin	My	Lin	Ful	Rea	—

Framework	16	32	64	128	256	512	Best	Cls	Lng	Plt	FE	Aos	DB	Dos	Orm	IA	Errors
aah	16,599	20,154	22,003	25,253	27,987	30,534	30,534	Ful	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
actframework-ebean-m	17,288	33,918	48,881	62,987	62,493	64,286	64,286	Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	0
actframework-ebean-m	2,365	1,506	867	332	110	47	2,365	Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	0
actframework-ebean-p	18,044	35,170	56,221	72,654	84,508	83,293	84,508	Ful	Jav	Utw	Non	Lin	Pg	Lin	Ful	Rea	0
actframework-ebean-p	17,301	10,972	0	0	0	0	17,301	Ful	Jav	Utw	Non	Lin	Pg	Lin	Ful	Rea	0
actframework-hiberna	15,410	30,513	45,275	52,242	56,241	51,412	56,241	Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	0
actframework-hiberna	18,362	35,082	47,433	45,999	42,610	33,385	47,433	Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	0
actframework-hiberna	16,859	32,522	52,068	68,642	77,573	77,964	77,964	Ful	Jav	Utw	Non	Lin	Pg	Lin	Ful	Rea	0
actframework-hiberna	20,941	38,733	44,625	42,335	47,067	36,663	47,067	Ful	Jav	Utw	Non	Lin	Pg	Lin	Ful	Rea	0
actframework-morphia	15,003	23,466	22,508	23,409	24,701	24,268	24,701	Ful	Jav	Utw	Non	Lin	Mo	Lin	Ful	Rea	0
actframework-morphia	17,612	21,738	19,101	18,432	18,468	18,532	21,738	Ful	Jav	Utw	Non	Lin	Mo	Lin	Ful	Rea	0
actframework-eclipse	— Did not complete —	Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	—
actframework-eclipse	— Did not complete —	Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	—
actframework-eclipse	— Did not complete —	Ful	Jav	Utw	Non	Lin	Pg	Lin	Ful	Rea	—
actframework-eclipse	— Did not complete —	Ful	Jav	Utw	Non	Lin	Pg	Lin	Ful	Rea	—
activeweb	4,916	8,194	13,201	13,986	12,450	10,860	13,986	Ful	Jav	Svt	Non	Lin	My	Lin	Mcr	Rea	0
aiohttp	6,809	12,321	21,930	27,123	30,705	31,528	31,528	Mcr	Py	asy	Gun	Lin	Pg	Lin	Ful	Rea	0
aiohttp-pg-raw	8,059	15,197	25,368	28,333	28,498	30,971	30,971	Mcr	Py	asy	Gun	Lin	Pg	Lin	Raw	Rea	0
akka-http	6,753	6,243	6,021	6,033	5,992	6,490	6,753	Mcr	Sca	Akk	Non	Lin	My	Lin	Raw	Rea	0
api_hour	6,388	12,791	22,098	28,264	30,337	28,292	30,337	Mcr	Py	asy	Gun	Lin	Pg	Lin	Raw	Rea	0
api_hour-mysql	5,064	10,464	17,676	23,236	25,192	26,068	26,068	Mcr	Py	asy	Gun	Lin	My	Lin	Raw	Rea	0
bottle	4,986	9,470	15,434	15,353	13,161	13,165	15,434	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
bottle-mysql-raw	11,236	22,788	40,225	45,787	43,642	40,819	45,787	Mcr	Py	Mei	Non	Lin	My	Lin	Raw	Rea	0
bottle-nginx-uwsgi	4,071	4,690	4,727	4,951	5,015	4,694	5,015	Mcr	Py	uWS	ngx	Lin	My	Lin	Ful	Rea	0
bottle-py3	5,028	9,575	16,450	16,990	16,259	15,766	16,990	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
bottle-pypy2	3,131	5,478	6,912	6,205	4,809	4,996	6,912	Mcr	Py	Tor	Non	Lin	My	Lin	Ful	Rea	0
cpoll_cppsp-raw	35,939	72,776	107,636	112,982	109,555	101,525	112,982	Plt	C++	Non	Non	Lin	My	Lin	Raw	Rea	0
cutelyst-nginx-mysql	18,426	41,438	73,184	73,206	73,194	73,149	73,206	Ful	C++	Qt	ngx	Lin	My	Lin	Raw	Rea	0
cutelyst-nginx-postg	22,306	45,096	100,941	84,807	84,782	85,436	100,941	Ful	C++	Qt	ngx	Lin	Pg	Lin	Raw	Rea	0
cutelyst-pf-mysql	10,311	17,940	34,606	56,829	76,043	76,323	76,323	Ful	C++	Qt	Non	Lin	My	Lin	Raw	Rea	0
cutelyst-pf-mysql-ep	11,933	26,968	28,295	42,892	60,412	51,691	60,412	Ful	C++	Qt	Non	Lin	My	Lin	Raw	Rea	0
cutelyst-pf-postgres	10,807	20,238	32,459	52,682	85,684	110,129	110,129	Ful	C++	Qt	Non	Lin	Pg	Lin	Raw	Rea	0
cutelyst-thread-mysq	23,138	52,966	77,530	93,943	97,649	98,720	98,720	Ful	C++	Qt	Non	Lin	My	Lin	Raw	Rea	0
cutelyst-thread-mysq	24,488	51,011	78,881	91,390	96,868	98,159	98,159	Ful	C++	Qt	Non	Lin	My	Lin	Raw	Rea	0
cutelyst-thread-post	29,105	60,988	109,111	139,668	158,796	167,932	167,932	Ful	C++	Qt	Non	Lin	Pg	Lin	Raw	Rea	0
cutelyst-thread-post	30,837	56,860	97,676	145,909	160,278	167,095	167,095	Ful	C++	Qt	Non	Lin	Pg	Lin	Raw	Rea	0
django	3,911	7,440	11,643	11,275	9,969	10,197	11,643	Ful	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
django-postgresql	4,041	8,204	12,673	12,347	11,498	11,417	12,673	Ful	Py	Non	Mei	Lin	Pg	Lin	Ful	Rea	0
django-py3	5,233	10,358	17,169	17,814	16,282	16,279	17,814	Ful	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
dropwizard-jdbi-post	17,668	28,884	30,512	31,255	31,744	30,781	31,744	Ful	Jav	JAX	Jty	Lin	Pg	Lin	Mcr	Rea	0
dropwizard-mongodb	9,136	10,187	10,491	10,431	12,339	11,989	12,339	Ful	Jav	JAX	Jty	Lin	Mo	Lin	Ful	Rea	0
dropwizard-mysql	14,571	25,902	29,103	32,419	34,314	35,510	35,510	Ful	Jav	JAX	Jty	Lin	My	Lin	Ful	Rea	0
dropwizard-postgres	16,015	27,452	30,700	31,166	33,729	32,467	33,729	Ful	Jav	JAX	Jty	Lin	Pg	Lin	Ful	Rea	0
falcore	11,372	18,186	30,717	36,343	35,629	38,055	38,055	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
ffead-cpp	0	0	0	0	0	0	0	Ful	C++	ffe	ffe	Lin	Mo	Lin	Ful	Rea	0
ffead-cpp-apache-mon	0	0	0	0	0	0	0	Ful	C++	ffe	Apc	Lin	Mo	Lin	Ful	Rea	0
ffead-cpp-apache-mys	0	0	0	0	0	0	0	Ful	C++	ffe	Apc	Lin	My	Lin	Ful	Rea	0
ffead-cpp-apache-pos	2,603	2,861	2,151	1,938	1,930	1,907	2,861	Ful	C++	ffe	Apc	Lin	Pg	Lin	Ful	Rea	0
ffead-cpp-mysql	0	0	0	0	0	0	0	Ful	C++	ffe	ffe	Lin	My	Lin	Ful	Rea	0
ffead-cpp-nginx-mong	0	0	0	0	0	0	0	Ful	C++	ffe	ngx	Lin	Mo	Lin	Ful	Rea	0
ffead-cpp-nginx-mysq	0	0	0	0	0	0	0	Ful	C++	ffe	ngx	Lin	My	Lin	Ful	Rea	0
ffead-cpp-nginx-post	2,559	2,847	2,128	1,941	1,929	1,909	2,847	Ful	C++	ffe	ngx	Lin	Pg	Lin	Ful	Rea	0
ffead-cpp-postgresql	0	0	0	0	0	0	0	Ful	C++	ffe	ffe	Lin	Pg	Lin	Ful	Rea	0
fintrospect	18,482	35,031	65,665	85,893	84,381	81,285	85,893	Mcr	Sca	Nty	Non	Lin	My	Lin	Raw	Rea	0
flask	4,684	8,612	12,580	10,705	10,415	10,740	12,580	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
flask-mysql-raw	8,307	15,742	26,198	25,019	19,887	21,017	26,198	Mcr	Py	Mei	Non	Lin	My	Lin	Raw	Rea	0
flask-nginx-uwsgi	3,887	4,829	4,971	5,147	5,084	5,658	5,658	Mcr	Py	Non	ngx	Lin	My	Lin	Ful	Rea	0
flask-py3	4,554	9,184	14,469	14,264	12,939	12,914	14,469	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
flask-pypy2	2,201	2,965	3,264	3,053	2,429	2,595	3,264	Mcr	Py	Non	Tor	Lin	My	Lin	Ful	Rea	0
flask-pypy2-mysql-ra	4,175	6,176	8,741	7,600	6,043	5,660	8,741	Mcr	Py	Tor	Non	Lin	My	Lin	Raw	Rea	0
gemini-mysql	24,788	47,544	80,662	108,138	103,497	53,841	108,138	Ful	Jav	Svt	Res	Lin	My	Lin	Mcr	Rea	0
gemini-postgres	31,820	67,137	117,426	173,928	161,412	121,503	173,928	Ful	Jav	Svt	Res	Lin	Pg	Lin	Mcr	Rea	0
gin	19,657	28,032	44,347	34,825	35,337	37,545	44,347	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
goji	19,422	29,216	53,466	47,656	38,764	22,324	53,466	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
grizzly-jersey	15,248	29,668	46,890	59,446	59,148	62,635	62,635	Mcr	Jav	JAX	Non	Lin	My	Lin	Ful	Rea	0
http4s	17,029	33,392	52,498	53,058	53,768	53,375	53,768	Mcr	Sca	NIO	bla	Lin	Pg	Lin	Raw	Rea	0
jawn	23,972	29,101	28,374	29,118	29,316	28,953	29,316	Ful	Jav	Svt	Utw	Lin	Pg	Lin	Raw	Rea	0
undertow-jersey-c3p0	6,854	13,518	17,594	17,393	16,345	15,393	17,594	Plt	Jav	JAX	Non	Lin	My	Lin	Ful	Rea	0
undertow-jersey-hika	9,982	16,715	19,035	18,340	17,363	16,733	19,035	Plt	Jav	JAX	Non	Lin	My	Lin	Ful	Rea	0
jooby	26,656	43,656	62,349	70,285	68,361	72,687	72,687	Ful	Jav	Nty	Non	Lin	Pg	Lin	Raw	Rea	0
kami	20,081	30,022	59,441	56,666	53,891	54,098	59,441	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
klein	266	265	265	263	255	250	266	Mcr	Py	Non	Twi	Lin	My	Lin	Ful	Rea	0
minijax	21,856	32,383	34,916	36,362	36,183	38,574	38,574	Ful	Jav	JAX	Jty	Lin	My	Lin	Ful	Rea	0
morepath	6,135	11,737	20,201	23,011	21,839	21,321	23,011	Mcr	Py	Mei	Gun	Lin	Pg	Lin	Ful	Rea	0
ninja-standalone	5,850	6,665	6,904	6,867	7,003	7,011	7,011	Ful	Jav	Jty	Non	Lin	My	Lin	Ful	Rea	0
cherrypy	196	197	195	193	194	181	197	Mcr	Py	Non	Non	Lin	My	Lin	Ful	Rea	0
cherrypy-py3	255	251	239	241	252	253	255	Mcr	Py	Non	Non	Lin	My	Lin	Ful	Rea	0
cppcms	14,434	15,091	14,698	14,864	14,654	14,695	15,091	Plt	C++	Non	Non	Lin	My	Lin	Raw	Rea	0
cppcms-direct	10,488	10,641	10,548	10,857	10,767	10,725	10,857	Plt	C++	Non	Non	Lin	My	Lin	Raw	Rea	0
cppcms-postgres	16,001	16,308	15,540	15,038	15,148	15,082	16,308	Plt	C++	Non	Non	Lin	Pg	Lin	Raw	Rea	0
echo	19,692	26,902	36,725	27,586	28,483	26,378	36,725	Mcr	Go	Non	Non	Lin	Pg	Lin	Raw	Rea	0
fasthttp	31,197	39,162	65,152	51,016	46,264	36,677	65,152	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
fasthttp-postgresql	33,416	71,507	77,884	101,400	124,514	89,580	124,514	Plt	Go	Non	Non	Lin	Pg	Lin	Raw	Rea	0
go	19,969	28,510	49,440	46,582	43,799	37,330	49,440	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
go-interpolate	19,468	28,434	51,216	49,991	49,835	48,657	51,216	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
go-mongo	10,864	10,738	10,500	10,297	10,271	10,224	10,864	Plt	Go	Non	Non	Lin	Mo	Lin	Raw	Rea	0
go-postgres	21,018	29,632	50,347	44,944	42,174	36,130	50,347	Plt	Go	Non	Non	Lin	Pg	Lin	Raw	Rea	0
go-prefork	21,537	40,105	71,266	94,602	106,627	112,122	112,122	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
light-4j-postgresql	31,393	46,981	81,379	153,618	161,177	167,583	167,583	Plt	Jav	Lig	Non	Lin	Pg	Lin	Raw	Rea	0
revel-jet	15,704	24,218	42,558	37,060	35,786	37,073	42,558	Ful	Go	Non	Non	Lin	My	Lin	Mcr	Rea	0
revel-qbs	16,510	24,081	38,816	34,680	33,862	35,423	38,816	Ful	Go	Non	Non	Lin	My	Lin	Mcr	Rea	0
revel-raw	18,023	26,106	44,742	43,873	40,199	42,727	44,742	Ful	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
servlet-postgres-raw	29,541	42,126	42,227	43,501	44,812	47,985	47,985	Plt	Jav	Svt	Res	Lin	Pg	Lin	Raw	Rea	0
servlet-raw	25,701	40,414	40,989	42,930	44,830	47,862	47,862	Plt	Jav	Svt	Res	Lin	My	Lin	Raw	Rea	0
tornado	4,413	8,062	11,813	12,672	12,449	12,352	12,672	Plt	Py	Non	Tor	Lin	Mo	Lin	Raw	Rea	0
tornado-py3	3,999	7,385	11,335	12,451	13,259	13,277	13,277	Plt	Py	Non	Tor	Lin	Mo	Lin	Raw	Rea	0
tornado-pypy2	12,206	22,042	35,038	37,987	40,210	38,015	40,210	Plt	Py	Non	Tor	Lin	Mo	Lin	Raw	Rea	0
undertow-mongodb	22,724	37,716	55,014	87,052	94,833	60,128	94,833	Plt	Jav	Utw	Non	Lin	Mo	Lin	Raw	Rea	0
undertow-mongodb-asy	22,392	43,285	43,218	45,093	44,353	45,602	45,602	Plt	Jav	Utw	Non	Lin	Mo	Lin	Raw	Rea	0
undertow-mysql	25,369	43,346	60,036	99,161	106,820	108,367	108,367	Plt	Jav	Utw	Non	Lin	My	Lin	Raw	Rea	0
undertow-postgresql	27,423	44,230	67,219	122,517	134,155	132,728	134,155	Plt	Jav	Utw	Non	Lin	Pg	Lin	Raw	Rea	0
vertx-postgres	28,399	52,330	99,983	146,000	124,616	134,478	146,000	Plt	Jav	Ver	Non	Lin	Pg	Lin	Raw	Rea	0
wildfly-ee7	8,124	13,959	15,657	14,656	14,940	14,972	15,657	Ful	Jav	Svt	Wil	Lin	My	Lin	Ful	Rea	0
play2-java-ebean-hik	9,975	13,937	13,909	15,376	13,923	15,022	15,376	Ful	Jav	Akk	Non	Lin	My	Lin	Ful	Rea	0
play2-java-ebean-hik	10,626	14,209	14,165	14,485	13,825	13,173	14,485	Ful	Jav	Nty	Non	Lin	My	Lin	Ful	Rea	0
play2-java-jooq-hika	12,587	18,337	18,027	18,739	18,074	18,968	18,968	Ful	Jav	Akk	Non	Lin	My	Lin	Ful	Rea	0
play2-java-jooq-hika	14,139	19,809	22,165	19,567	19,172	19,037	22,165	Ful	Jav	Nty	Non	Lin	My	Lin	Ful	Rea	0
play2-java-jpa-hikar	14,982	23,636	24,981	24,406	24,881	24,884	24,981	Ful	Jav	Akk	Non	Lin	My	Lin	Ful	Rea	0
play2-java-jpa-hikar	15,940	25,574	25,937	26,798	25,512	25,002	26,798	Ful	Jav	Nty	Non	Lin	My	Lin	Ful	Rea	0
play2-scala-anorm-li	14,843	11,008	13,253	13,252	13,405	13,732	14,843	Ful	Sca	Akk	Non	Lin	My	Lin	Ful	Rea	0
play2-scala-anorm-li	16,898	12,600	12,714	13,111	12,713	12,749	16,898	Ful	Sca	Nty	Non	Lin	My	Lin	Ful	Rea	0
play2-scala-reactive	5,330	7,429	11,823	11,727	12,740	12,057	12,740	Ful	Sca	Akk	Non	Lin	Mo	Lin	Ful	Rea	0
play2-scala-reactive	10,030	12,937	19,420	17,941	20,649	16,880	20,649	Ful	Sca	Nty	Non	Lin	Mo	Lin	Ful	Rea	0
play2-scala-slick	14,476	19,990	19,550	19,349	19,064	19,574	19,990	Ful	Sca	Akk	Non	Lin	My	Lin	Ful	Rea	0
play2-scala-slick-ne	15,214	21,378	19,787	19,014	18,614	18,493	21,378	Ful	Sca	Nty	Non	Lin	My	Lin	Ful	Rea	0
proteus	32,704	48,484	85,940	136,691	139,414	139,484	139,484	Mcr	Jav	Utw	Non	Lin	Pg	Lin	Raw	Rea	0
proteus-mysql	32,424	49,621	90,221	152,351	162,284	162,613	162,613	Mcr	Jav	Utw	Non	Lin	My	Lin	Raw	Rea	0
pyramid	6,346	12,359	20,603	20,290	18,531	18,539	20,603	Ful	Py	Non	Mei	Lin	Pg	Lin	Ful	Rea	0
pyramid-py2	6,642	12,846	21,138	19,703	16,751	16,751	21,138	Ful	Py	Non	Mei	Lin	Pg	Lin	Ful	Rea	0
rapidoid-mysql	23,556	52,465	92,031	109,026	117,301	117,107	117,301	Plt	Jav	Rap	Non	Lin	My	Lin	Mcr	Rea	0
rapidoid-postgres	25,648	67,025	102,198	108,082	109,892	111,093	111,093	Plt	Jav	Rap	Non	Lin	Pg	Lin	Mcr	Rea	0
revenj-jvm	30,194	57,983	87,123	82,758	67,240	52,703	87,123	Ful	Jav	Svt	Res	Lin	Pg	Lin	Ful	Rea	0
silicon	25,288	47,827	67,538	69,547	66,992	62,585	69,547	Mcr	C++	Non	mic	Lin	My	Lin	Ful	Rea	0
spark	194	224	219	215	206	201	224	Mcr	Jav	Svt	Res	Lin	My	Lin	Ful	Rea	0
spring	6,073	12,003	19,062	20,292	19,996	22,439	22,439	Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	0
tapestry	9,107	14,831	24,654	29,473	28,313	26,787	29,473	Ful	Jav	Svt	Res	Lin	My	Lin	Ful	Rea	0
treefrog	13,817	28,812	44,179	55,606	50,527	46,721	55,606	Ful	C++	Non	Non	Lin	My	Lin	Mcr	Rea	0
treefrog-hybrid	8,423	18,053	27,502	36,304	42,699	46,316	46,316	Ful	C++	Non	Non	Lin	My	Lin	Mcr	Rea	8
treefrog-mongodb	19,848	37,597	64,986	95,275	111,614	115,114	115,114	Ful	C++	Non	Non	Lin	Mo	Lin	Mcr	Rea	0
treefrog-postgres	12,459	27,683	45,604	56,352	51,351	43,866	56,352	Ful	C++	Non	Non	Lin	Pg	Lin	Mcr	Rea	0
turbogears	4,902	9,462	14,157	11,762	11,165	11,173	14,157	Mcr	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
ulib-mongodb	29,302	64,934	111,629	147,204	165,844	174,785	174,785	Plt	C++	Non	ULi	Lin	Mo	Lin	Mcr	Rea	0
ulib-mysql	39,791	80,947	124,855	136,976	136,770	136,291	136,976	Plt	C++	Non	ULi	Lin	My	Lin	Mcr	Rea	0
ulib-postgres	42,622	97,306	140,860	184,679	184,205	183,955	184,679	Plt	C++	Non	ULi	Lin	Pg	Lin	Mcr	Rea	0
ulib-postgres_fit	44,201	91,601	137,181	155,126	185,279	189,705	189,705	Plt	C++	Non	ULi	Lin	Pg	Lin	Mcr	Rea	0
uvicorn	17,702	30,685	31,768	32,809	32,672	32,991	32,991	Plt	Py	Non	Non	Lin	Pg	Lin	Raw	Rea	0
vertx-web-mongodb	22,789	40,172	42,615	42,175	42,000	44,949	44,949	Mcr	Jav	vtx	Non	Lin	Mo	Lin	Raw	Rea	0
vertx-web-postgres	25,635	44,683	47,214	43,786	44,902	44,398	47,214	Mcr	Jav	vtx	Non	Lin	Pg	Lin	Raw	Rea	0
vertx-web-susom-post	6,509	10,969	18,720	34,022	39,877	38,446	39,877	Mcr	Jav	vtx	Non	Lin	Pg	Lin	Raw	Rea	0
web2py	2,288	4,210	6,285	6,326	6,019	6,075	6,326	Ful	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
web2py-optimized	4,012	7,581	11,141	11,262	10,529	10,756	11,262	Ful	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
webware	0	0	0	0	0	0	0	Mcr	Py	Non	Non	Lin	My	Lin	Ful	Rea	1,154
weppy	5,367	10,407	17,660	20,183	18,306	16,862	20,183	Ful	Py	Mei	Non	Lin	Pg	Lin	Ful	Rea	0
weppy-nginx-uwsgi	4,297	4,817	4,962	5,123	4,986	4,955	5,123	Ful	Py	uWS	ngx	Lin	Pg	Lin	Ful	Rea	0
weppy-py3	5,369	10,317	18,075	21,806	20,691	20,719	21,806	Ful	Py	Mei	Non	Lin	Pg	Lin	Ful	Rea	0
weppy-pypy2	1,928	2,557	3,132	2,913	2,140	2,189	3,132	Ful	Py	Tor	Non	Lin	Pg	Lin	Ful	Rea	0
wheezyweb	7,999	16,158	27,000	27,862	23,365	24,145	27,862	Mcr	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
wheezyweb-py3	— Did not complete —	Mcr	Py	Non	Mei	Lin	My	Lin	Ful	Rea	—
wicket	13,093	20,306	20,493	20,537	21,168	19,742	21,168	Ful	Jav	Svt	Res	Lin	My	Lin	Ful	Rea	0
wt	12,252	12,156	20,084	30,868	30,672	30,869	30,869	Ful	C++	Non	Non	Lin	My	Lin	Ful	Rea	0
wt-postgres	12,767	12,403	23,957	30,045	29,966	30,037	30,045	Ful	C++	Non	Non	Lin	Pg	Lin	Ful	Rea	0

Framework	Average latency (lower is better)	σ (SD)	Max	Errors
ulib-postgres_fit	2.7 ms	0.1%	1.2 ms	60.5 ms	0
ulib-postgres	2.9 ms	0.1%	1.7 ms	96.7 ms	0
ulib-mongodb	3.0 ms	0.1%	1.4 ms	26.1 ms	0
cutelyst-thread-post	3.1 ms	0.1%	1.4 ms	34.6 ms	0
cutelyst-thread-post	3.1 ms	0.1%	1.4 ms	30.4 ms	0
proteus-mysql	3.7 ms	0.1%	4.0 ms	212.2 ms	0
light-4j-postgresql	3.9 ms	0.1%	9.1 ms	218.9 ms	0
ulib-mysql	3.9 ms	0.1%	2.1 ms	58.6 ms	0
vertx-web-susom-post	4.0 ms	0.1%	5.2 ms	86.5 ms	0
undertow-postgresql	4.7 ms	0.1%	6.9 ms	128.8 ms	0
cutelyst-pf-postgres	4.7 ms	0.1%	1.9 ms	25.1 ms	0
vertx-postgres	5.2 ms	0.1%	10.2 ms	466.1 ms	0
cutelyst-thread-mysq	5.3 ms	0.1%	2.6 ms	40.8 ms	0
cutelyst-thread-mysq	5.3 ms	0.1%	2.5 ms	35.8 ms	0
cutelyst-nginx-postg	6.0 ms	0.1%	0.8 ms	27.8 ms	0
proteus	6.1 ms	0.1%	9.6 ms	104.5 ms	0
rapidoid-postgres	6.7 ms	0.1%	11.5 ms	218.9 ms	0
go-prefork	6.8 ms	0.1%	16.7 ms	348.9 ms	0
cutelyst-pf-mysql	6.8 ms	0.1%	2.5 ms	31.7 ms	0
cutelyst-nginx-mysql	7.0 ms	0.1%	0.3 ms	21.6 ms	0
fasthttp-postgresql	7.2 ms	0.1%	8.4 ms	246.8 ms	0
actframework-ebean-p	7.2 ms	0.1%	9.3 ms	235.2 ms	0
rapidoid-mysql	7.3 ms	0.1%	11.8 ms	109.6 ms	0
actframework-hiberna	7.7 ms	0.1%	7.8 ms	216.0 ms	0
treefrog-mongodb	7.7 ms	0.1%	29.6 ms	821.3 ms	0
grizzly-jersey	8.5 ms	0.2%	6.6 ms	147.9 ms	0
undertow-mysql	8.6 ms	0.2%	13.5 ms	238.3 ms	0
jooby	8.8 ms	0.2%	11.7 ms	214.6 ms	0
actframework-ebean-m	8.9 ms	0.2%	7.1 ms	139.1 ms	0
actframework-hiberna	11.3 ms	0.2%	8.7 ms	116.7 ms	0
kami	11.3 ms	0.2%	10.7 ms	113.5 ms	0
go-interpolate	12.0 ms	0.2%	10.8 ms	118.3 ms	0
cutelyst-pf-mysql-ep	13.0 ms	0.3%	14.4 ms	182.8 ms	0
revel-raw	13.1 ms	0.3%	12.8 ms	252.7 ms	0
api_hour	13.2 ms	0.3%	16.4 ms	452.5 ms	0
treefrog-hybrid	13.2 ms	0.3%	30.4 ms	856.0 ms	8
vertx-web-postgres	13.4 ms	0.3%	14.7 ms	244.6 ms	0
vertx-web-mongodb	14.1 ms	0.3%	18.2 ms	241.0 ms	0
bottle-mysql-raw	14.4 ms	0.3%	15.5 ms	277.6 ms	0
go-postgres	14.6 ms	0.3%	10.8 ms	94.5 ms	0
go	14.8 ms	0.3%	12.2 ms	266.2 ms	0
revel-jet	15.2 ms	0.3%	12.2 ms	124.8 ms	0
revel-qbs	15.4 ms	0.3%	12.3 ms	113.6 ms	0
gin	15.4 ms	0.3%	13.3 ms	152.3 ms	0
falcore	15.6 ms	0.3%	14.0 ms	149.8 ms	0
tornado-pypy2	15.7 ms	0.3%	18.7 ms	453.7 ms	0
servlet-raw	15.8 ms	0.3%	19.5 ms	234.2 ms	0
http4s	15.9 ms	0.3%	58.3 ms	1600.0 ms	0
servlet-postgres-raw	16.1 ms	0.3%	21.4 ms	256.4 ms	0
cpoll_cppsp-raw	16.9 ms	0.3%	77.5 ms	1370.0 ms	0
undertow-mongodb-asy	17.1 ms	0.3%	29.2 ms	338.8 ms	0
aiohttp	17.2 ms	0.3%	17.4 ms	427.1 ms	0
dropwizard-jdbi-post	17.2 ms	0.3%	10.4 ms	247.6 ms	0
uvicorn	17.3 ms	0.3%	21.8 ms	680.6 ms	0
fintrospect	17.6 ms	0.3%	92.4 ms	1620.0 ms	0
dropwizard-mysql	17.7 ms	0.3%	19.0 ms	296.3 ms	0
fasthttp	17.8 ms	0.3%	21.0 ms	293.9 ms	0
api_hour-mysql	18.0 ms	0.4%	14.0 ms	256.5 ms	0
aah	18.3 ms	0.4%	13.8 ms	261.0 ms	0
aiohttp-pg-raw	18.7 ms	0.4%	20.9 ms	464.1 ms	0
dropwizard-postgres	19.2 ms	0.4%	18.2 ms	197.6 ms	0
jawn	20.6 ms	0.4%	19.5 ms	224.6 ms	0
echo	21.1 ms	0.4%	19.7 ms	271.6 ms	0
revenj-jvm	22.0 ms	0.4%	31.3 ms	357.4 ms	0
actframework-morphia	22.0 ms	0.4%	13.8 ms	181.0 ms	0
tapestry	22.6 ms	0.4%	22.6 ms	252.8 ms	0
play2-java-jpa-hikar	22.9 ms	0.4%	20.9 ms	264.1 ms	0
spring	23.3 ms	0.5%	10.3 ms	112.5 ms	0
wt	23.4 ms	0.5%	60.5 ms	1130.0 ms	0
goji	23.9 ms	0.5%	17.3 ms	133.6 ms	0
wheezyweb	23.9 ms	0.5%	23.6 ms	514.0 ms	0
treefrog	24.6 ms	0.5%	75.6 ms	1640.0 ms	0
play2-java-jpa-hikar	24.7 ms	0.5%	41.9 ms	853.4 ms	0
weppy-py3	25.1 ms	0.5%	19.9 ms	527.5 ms	0
morepath	25.6 ms	0.5%	21.5 ms	699.6 ms	0
flask-mysql-raw	27.3 ms	0.5%	24.7 ms	432.9 ms	0
wt-postgres	28.4 ms	0.6%	93.8 ms	1520.0 ms	0
minijax	28.9 ms	0.6%	116.3 ms	1620.0 ms	0
pyramid	29.8 ms	0.6%	26.1 ms	609.8 ms	0
wicket	30.0 ms	0.6%	29.6 ms	356.7 ms	0
actframework-morphia	30.3 ms	0.6%	24.7 ms	407.8 ms	0
weppy	32.7 ms	0.6%	26.7 ms	450.8 ms	0
pyramid-py2	33.6 ms	0.7%	31.9 ms	972.5 ms	0
django-py3	33.9 ms	0.7%	28.4 ms	441.9 ms	0
cppcms-postgres	33.9 ms	0.7%	4.3 ms	48.6 ms	0
bottle-py3	34.4 ms	0.7%	27.7 ms	488.8 ms	0
cppcms	34.8 ms	0.7%	3.0 ms	46.3 ms	0
play2-java-ebean-hik	35.6 ms	0.7%	30.6 ms	843.8 ms	0
play2-java-jooq-hika	36.9 ms	0.7%	53.7 ms	492.1 ms	0
tornado-py3	37.9 ms	0.7%	17.9 ms	585.8 ms	0
play2-scala-anorm-li	38.6 ms	0.8%	29.0 ms	844.4 ms	0
cherrypy-py3	39.4 ms	0.8%	7.4 ms	261.3 ms	0
bottle	40.7 ms	0.8%	28.9 ms	497.6 ms	0
tornado	41.6 ms	0.8%	20.1 ms	520.9 ms	0
wildfly-ee7	41.9 ms	0.8%	42.0 ms	404.1 ms	0
flask-py3	42.3 ms	0.8%	33.6 ms	424.7 ms	0
play2-java-ebean-hik	42.8 ms	0.8%	29.4 ms	414.3 ms	0
play2-scala-anorm-li	43.7 ms	0.9%	28.3 ms	337.0 ms	0
undertow-jersey-hika	44.5 ms	0.9%	64.7 ms	491.4 ms	0
undertow-jersey-c3p0	44.8 ms	0.9%	63.0 ms	757.2 ms	0
silicon	45.2 ms	0.9%	244.2 ms	3270.0 ms	0
cppcms-direct	45.7 ms	0.9%	7.3 ms	244.3 ms	0
turbogears	48.0 ms	0.9%	37.5 ms	490.2 ms	0
django-postgresql	48.1 ms	0.9%	39.8 ms	661.4 ms	0
web2py-optimized	48.8 ms	1.0%	37.0 ms	672.1 ms	0
play2-java-jooq-hika	49.4 ms	1.0%	101.0 ms	851.2 ms	0
dropwizard-mongodb	49.9 ms	1.0%	46.2 ms	327.6 ms	0
go-mongo	50.0 ms	1.0%	2.3 ms	81.7 ms	0
flask	50.3 ms	1.0%	39.4 ms	646.6 ms	0
play2-scala-reactive	50.8 ms	1.0%	70.2 ms	1020.0 ms	0
gemini-mysql	51.1 ms	1.0%	104.9 ms	994.6 ms	0
django	52.7 ms	1.0%	41.1 ms	585.7 ms	0
cherrypy	55.0 ms	1.1%	12.9 ms	284.3 ms	0
play2-scala-slick-ne	56.1 ms	1.1%	75.3 ms	656.7 ms	0
treefrog-postgres	58.2 ms	1.1%	284.6 ms	3530.0 ms	0
undertow-mongodb	61.2 ms	1.2%	126.1 ms	759.0 ms	0
activeweb	67.3 ms	1.3%	80.3 ms	1040.0 ms	0
ninja-standalone	81.6 ms	1.6%	102.8 ms	1740.0 ms	0
web2py	87.8 ms	1.7%	63.1 ms	819.3 ms	0
flask-nginx-uwsgi	90.6 ms	1.8%	24.1 ms	331.9 ms	0
weppy-nginx-uwsgi	103.3 ms	2.0%	17.6 ms	347.4 ms	0
play2-scala-reactive	104.1 ms	2.0%	243.9 ms	2570.0 ms	0
flask-pypy2-mysql-ra	108.3 ms	2.1%	122.1 ms	2170.0 ms	0
bottle-nginx-uwsgi	109.0 ms	2.1%	17.7 ms	352.9 ms	0
gemini-postgres	118.2 ms	2.3%	232.3 ms	1970.0 ms	0
bottle-pypy2	133.5 ms	2.6%	173.0 ms	2500.0 ms	0
play2-scala-slick	148.2 ms	2.9%	508.3 ms	4650.0 ms	0
actframework-hiberna	210.7 ms	4.1%	574.4 ms	2990.0 ms	0
flask-pypy2	213.4 ms	4.2%	185.2 ms	2360.0 ms	0
weppy-pypy2	242.8 ms	4.7%	217.8 ms	2320.0 ms	0
actframework-hiberna	250.3 ms	4.9%	663.0 ms	3260.0 ms	0
ffead-cpp-nginx-post	256.3 ms	5.0%	43.0 ms	347.4 ms	0
ffead-cpp-apache-pos	266.1 ms	5.2%	26.0 ms	476.1 ms	0
akka-http	340.8 ms	6.7%	1020.0 ms	7700.0 ms	0
klein	1640.0 ms	32.0%	672.1 ms	4870.0 ms	0
actframework-ebean-m	1820.0 ms	35.5%	2030.0 ms	5890.0 ms	0
spark	1920.0 ms	37.5%	1920.0 ms	7860.0 ms	0
webware	5120.0 ms	100.0%	1450.0 ms	6990.0 ms	1,154

Framework best	vs	Platform best	Ratio (higher is better)
play2-java-jpa-hikar	24,981	vs	akka-http	6,753	369.9%
tornado-pypy2	40,210	vs	tornado	12,672	317.3%
play2-scala-slick	19,990	vs	akka-http	6,753	296.0%
play2-java-jooq-hika	18,968	vs	akka-http	6,753	280.9%
fasthttp-postgresql	124,514	vs	go	49,440	251.8%
play2-java-ebean-hik	15,376	vs	akka-http	6,753	227.7%
go-prefork	112,122	vs	go	49,440	226.8%
play2-scala-anorm-li	14,843	vs	akka-http	6,753	219.8%
treefrog-mongodb	115,114	vs	treefrog	55,606	207.0%
play2-scala-reactive	12,740	vs	akka-http	6,753	188.7%
fasthttp	65,152	vs	go	49,440	131.8%
kami	59,441	vs	go	49,440	120.2%
undertow-jersey-hika	19,035	vs	undertow-jersey-c3p0	17,594	108.2%
goji	53,466	vs	go	49,440	108.1%
tornado-py3	13,277	vs	tornado	12,672	104.8%
go-interpolate	51,216	vs	go	49,440	103.6%
go-postgres	50,347	vs	go	49,440	101.8%
servlet-postgres-raw	47,985	vs	servlet-raw	47,862	100.3%
wt-postgres	30,045	vs	wt	30,869	97.3%
undertow-jersey-c3p0	17,594	vs	undertow-jersey-hika	19,035	92.4%
revel-raw	44,742	vs	go	49,440	90.5%
gin	44,347	vs	go	49,440	89.7%
revel-jet	42,558	vs	go	49,440	86.1%
treefrog-hybrid	46,316	vs	treefrog	55,606	83.3%
revel-qbs	38,816	vs	go	49,440	78.5%
falcore	38,055	vs	go	49,440	77.0%
echo	36,725	vs	go	49,440	74.3%
dropwizard-mysql	35,510	vs	servlet-raw	47,862	74.2%
dropwizard-postgres	33,729	vs	servlet-postgres-raw	47,985	70.3%
dropwizard-jdbi-post	31,744	vs	servlet-postgres-raw	47,985	66.2%
aah	30,534	vs	go	49,440	61.8%
actframework-hiberna	47,067	vs	actframework-hiberna	77,964	60.4%
go-mongo	10,864	vs	go	49,440	22.0%

Best (bar chart)

Data table

Latency

Framework overhead

Best (bar chart)

Data table

Latency

Framework overhead

Requirements summary

In this test, the framework's ORM is used to fetch all rows from a database table containing an unknown number of Unix fortune cookie messages (the table has 12 rows, but the code cannot have foreknowledge of the table's size). An additional fortune cookie message is inserted into the list at runtime and then the list is sorted by the message text. Finally, the list is delivered to the client using a server-side HTML template. The message text must be considered untrusted and properly escaped and the UTF-8 fortune messages must be rendered properly.

Whitespace is optional and may comply with the framework's best practices.

Example response:

HTTP/1.1 200 OK Content-Length: 1196 Content-Type: text/html; charset=UTF-8 Server: Example Date: Wed, 17 Apr 2013 12:00:00 GMT <!DOCTYPE html><html><head><title>Fortunes</title></head><body><table><tr><th>id</th><th>message</th></tr><tr><td>11</td><td><script>alert("This should not be displayed in a browser alert box.");</script></td></tr><tr><td>4</td><td>A bad random number generator: 1, 1, 1, 1, 1, 4.33e+67, 1, 1, 1</td></tr><tr><td>5</td><td>A computer program does what you tell it to do, not what you want it to do.</td></tr><tr><td>2</td><td>A computer scientist is someone who fixes things that aren't broken.</td></tr><tr><td>8</td><td>A list is only as strong as its weakest link. — Donald Knuth</td></tr><tr><td>0</td><td>Additional fortune added at request time.</td></tr><tr><td>3</td><td>After enough decimal places, nobody gives a damn.</td></tr><tr><td>7</td><td>Any program that runs right is obsolete.</td></tr><tr><td>10</td><td>Computers make very fast, very accurate mistakes.</td></tr><tr><td>6</td><td>Emacs is a nice operating system, but I prefer UNIX. — Tom Christaensen</td></tr><tr><td>9</td><td>Feature: A bug with seniority.</td></tr><tr><td>1</td><td>fortune: No such file or directory</td></tr><tr><td>12</td><td>フレームワークのベンチマーク</td></tr></table></body></html>

For a more detailed description of the requirements, see the Source Code and Requirements section.

Best JSON responses per second, Dell servers at ServerCentral(138 tests)

Framework	Best performance (higher is better)		Cls	Lng	Plt	FE	Aos	DB	Dos	Orm	IA	Errors
ulib-json	682,383	100.0%	Plt	C++	Non	ULi	Lin	Non	Lin	Mcr	Rea	0
revenj-jvm	669,526	98.1%	Ful	Jav	Svt	Res	Lin	Pg	Lin	Ful	Rea	0
ulib-json_fit	664,458	97.4%	Plt	C++	Non	ULi	Lin	Non	Lin	Mcr	Rea	0
servlet-cjs	663,623	97.3%	Plt	Jav	Svt	Res	Lin	Non	Lin	Raw	Rea	0
rapidoid	662,269	97.1%	Plt	Jav	Rap	Non	Lin	Non	Lin	Raw	Rea	0
servlet-afterburner	662,134	97.0%	Plt	Jav	Svt	Res	Lin	Non	Lin	Raw	Rea	0
proteus	658,410	96.5%	Mcr	Jav	Utw	Non	Lin	Pg	Lin	Raw	Rea	0
servlet	657,791	96.4%	Plt	Jav	Svt	Res	Lin	Non	Lin	Raw	Rea	0
cpoll_cppsp	646,776	94.8%	Plt	C++	Non	Non	Lin	Non	Lin	Raw	Rea	0
rapidoid-http-fast	642,181	94.1%	Plt	Jav	Rap	Non	Lin	Non	Lin	Raw	Rea	0
grizzly	641,988	94.1%	Mcr	Jav	Svt	Non	Lin	Non	Lin	Raw	Rea	0
light-4j	634,889	93.0%	Plt	Jav	Lig	Non	Lin	Non	Lin	Raw	Rea	0
s-server	633,811	92.9%	Plt	Sca	s-s	Non	Lin	Non	Lin	Raw	Rea	0
japronto	629,566	92.3%	Mcr	Py	Non	Non	Lin	Non	Lin	Raw	Rea	0
blaze	617,537	90.5%	Mcr	Sca	bla	Non	Lin	Non	Lin	Raw	Rea	0
cutelyst-thread	611,406	89.6%	Ful	C++	Qt	Non	Lin	Non	Lin	Raw	Rea	0
go-prefork	603,592	88.5%	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
undertow	595,431	87.3%	Plt	Jav	Utw	Non	Lin	Non	Lin	Raw	Rea	0
netty	590,560	86.5%	Plt	Jav	Nty	Non	Lin	Non	Lin	Raw	Rea	0
apistar	575,507	84.3%	Mcr	Py	Non	Mei	Lin	Non	Lin	Raw	Rea	0
vertx-web	570,243	83.6%	Mcr	Jav	vtx	Non	Lin	Non	Lin	Raw	Rea	0
cutelyst-thread-epol	559,997	82.1%	Ful	C++	Qt	Non	Lin	Non	Lin	Raw	Rea	0
vertx	550,381	80.7%	Plt	Jav	Ver	Non	Lin	Non	Lin	Raw	Rea	0
finatra	523,712	76.7%	Mcr	Sca	Nty	Non	Lin	Non	Lin	Raw	Rea	0
fintrospect	504,706	74.0%	Mcr	Sca	Nty	Non	Lin	My	Lin	Raw	Rea	0
gemini	498,666	73.1%	Ful	Jav	Svt	Res	Lin	Non	Lin	Mcr	Rea	0
colossus	497,039	72.8%	Mcr	Sca	Akk	Non	Lin	Non	Lin	Raw	Rea	0
falcon	477,082	69.9%	Mcr	Py	Non	Mei	Lin	Non	Lin	Raw	Rea	0
uvicorn	475,636	69.7%	Plt	Py	Non	Non	Lin	Pg	Lin	Raw	Rea	0
cutelyst-thread-epol	461,957	67.7%	Ful	C++	Qt	Non	Lin	Non	Lin	Raw	Rea	0
falcon-py3	424,771	62.2%	Mcr	Py	Non	Mei	Lin	Non	Lin	Raw	Rea	0
finagle	420,556	61.6%	Mcr	Sca	Nty	Non	Lin	Non	Lin	Raw	Rea	0
finch	396,796	58.1%	Mcr	Sca	Nty	Non	Lin	Non	Lin	Raw	Rea	0
actframework	389,125	57.0%	Ful	Jav	Utw	Non	Lin	Non	Lin	Ful	Rea	0
wheezyweb	387,714	56.8%	Mcr	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
bottle	364,428	53.4%	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
jlhttp	360,176	52.8%	Plt	Jav	JLH	Non	Lin	Non	Lin	Raw	Rea	0
fasthttp	357,675	52.4%	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
blade	327,933	48.1%	Ful	Jav	Non	Non	Lin	My	Lin	Ful	Rea	0
kami	309,291	45.3%	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
treefrog-mongodb	306,788	45.0%	Ful	C++	Non	Non	Lin	Mo	Lin	Mcr	Rea	0
treefrog	304,265	44.6%	Ful	C++	Non	Non	Lin	My	Lin	Mcr	Rea	0
bottle-py3	298,316	43.7%	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
go	296,356	43.4%	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
treefrog-postgres	293,190	43.0%	Ful	C++	Non	Non	Lin	Pg	Lin	Mcr	Rea	0
grizzly-jersey	286,502	42.0%	Mcr	Jav	JAX	Non	Lin	My	Lin	Ful	Rea	0
wicket	283,711	41.6%	Ful	Jav	Svt	Res	Lin	My	Lin	Ful	Rea	0
gin	282,655	41.4%	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
beego	271,747	39.8%	Ful	Go	Non	Non	Lin	Non	Lin	Raw	Rea	0
weppy	249,253	36.5%	Ful	Py	Mei	Non	Lin	Pg	Lin	Ful	Rea	0
falcore	229,102	33.6%	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
spark	225,691	33.1%	Mcr	Jav	Svt	Res	Lin	My	Lin	Ful	Rea	0
sanic	219,254	32.1%	Mcr	Py	Non	Non	Lin	Non	Lin	Raw	Rea	0
morepath	219,172	32.1%	Mcr	Py	Mei	Gun	Lin	Pg	Lin	Ful	Rea	0
weppy-py3	218,228	32.0%	Ful	Py	Mei	Non	Lin	Pg	Lin	Ful	Rea	0
play2-scala-scala-ne	208,400	30.5%	Ful	Sca	Nty	Non	Lin	Non	Lin	Raw	Rea	0
falcon-pypy2	199,972	29.3%	Mcr	Py	Non	Tor	Lin	Non	Lin	Raw	Rea	0
tornado-pypy2	196,384	28.8%	Plt	Py	Non	Tor	Lin	Mo	Lin	Raw	Rea	0
aiohttp	182,406	26.7%	Mcr	Py	asy	Gun	Lin	Pg	Lin	Ful	Rea	0
jetty-servlet	182,147	26.7%	Plt	Jav	Jty	Non	Lin	Non	Lin	Raw	Rea	0
jetty	177,118	26.0%	Plt	Jav	Jty	Non	Lin	Non	Lin	Raw	Rea	0
play2-java-netty	176,509	25.9%	Ful	Jav	Nty	Non	Lin	Non	Lin	Raw	Rea	0
pyramid-py2	175,201	25.7%	Ful	Py	Non	Mei	Lin	Pg	Lin	Ful	Rea	0
jawn	162,837	23.9%	Ful	Jav	Svt	Utw	Lin	Pg	Lin	Raw	Rea	0
activeweb-jackson	161,550	23.7%	Ful	Jav	Svt	Non	Lin	My	Lin	Mcr	Rea	0
pyramid	157,687	23.1%	Ful	Py	Non	Mei	Lin	Pg	Lin	Ful	Rea	0
silicon	156,779	23.0%	Mcr	C++	Non	mic	Lin	My	Lin	Ful	Rea	0
bayou	152,958	22.4%	Plt	Jav	Bay	Non	Lin	Non	Lin	Raw	Rea	0
spray	148,368	21.7%	Mcr	Sca	Akk	Non	Lin	Non	Lin	Raw	Rea	0
tapestry	146,931	21.5%	Ful	Jav	Svt	Res	Lin	My	Lin	Ful	Rea	0
undertow-jersey-hika	146,497	21.5%	Plt	Jav	JAX	Non	Lin	My	Lin	Ful	Rea	0
django-py3	146,354	21.4%	Ful	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
undertow-jersey-c3p0	145,981	21.4%	Plt	Jav	JAX	Non	Lin	My	Lin	Ful	Rea	0
echo	139,626	20.5%	Mcr	Go	Non	Non	Lin	Pg	Lin	Raw	Rea	0
webgo	139,313	20.4%	Mcr	Go	Non	Non	Lin	Non	Lin	Raw	Rea	0
restexpress	115,937	17.0%	Mcr	Jav	Nty	Non	Lin	Mo	Lin	Raw	Rea	0
revel	112,802	16.5%	Ful	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
curacao	109,172	16.0%	Mcr	Jav	Svt	Non	Lin	Non	Lin	Raw	Rea	5
akka-http	102,259	15.0%	Mcr	Sca	Akk	Non	Lin	My	Lin	Raw	Rea	0
jooby	101,330	14.8%	Ful	Jav	Nty	Non	Lin	Pg	Lin	Raw	Rea	0
api_hour	100,836	14.8%	Mcr	Py	asy	Gun	Lin	Pg	Lin	Raw	Rea	0
cutelyst-nginx	98,740	14.5%	Ful	C++	Qt	ngx	Lin	Non	Lin	Raw	Rea	0
comsat-webactors-und	97,801	14.3%	Plt	Jav	Com	Non	Lin	Non	Lin	Raw	Rea	0
flask-py3	93,918	13.8%	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
cutelyst-pf-epoll	93,555	13.7%	Ful	C++	Qt	Non	Lin	Non	Lin	Raw	Rea	0
cutelyst	93,025	13.6%	Ful	C++	Qt	Non	Lin	Non	Lin	Raw	Rea	0
play2-scala	93,002	13.6%	Ful	Sca	Akk	Non	Lin	Non	Lin	Raw	Rea	0
turbogears	86,535	12.7%	Mcr	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
flask	83,230	12.2%	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
http4s	82,652	12.1%	Mcr	Sca	NIO	bla	Lin	Pg	Lin	Raw	Rea	0
goji	82,014	12.0%	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
django	77,278	11.3%	Ful	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
comsat-webactors	74,246	10.9%	Plt	Jav	Com	Non	Lin	Non	Lin	Raw	Rea	0
wildfly-ee7	70,103	10.3%	Ful	Jav	Svt	Wil	Lin	My	Lin	Ful	Rea	0
treefrog-hybrid	69,681	10.2%	Ful	C++	Non	Non	Lin	My	Lin	Mcr	Rea	18
weppy-pypy2	62,790	9.2%	Ful	Py	Tor	Non	Lin	Pg	Lin	Ful	Rea	0
wt	59,385	8.7%	Ful	C++	Non	Non	Lin	My	Lin	Ful	Rea	0
tornado	59,293	8.7%	Plt	Py	Non	Tor	Lin	Mo	Lin	Raw	Rea	0
play2	58,043	8.5%	Ful	Jav	Akk	Non	Lin	Non	Lin	Raw	Rea	0
activeweb	57,299	8.4%	Ful	Jav	Svt	Non	Lin	My	Lin	Mcr	Rea	0
comsat-servlet	55,583	8.1%	Plt	Jav	Svt	Non	Lin	Non	Lin	Raw	Rea	0
play1	55,266	8.1%	Ful	Jav	Nty	Non	Lin	My	Lin	Ful	Rea	0
scruffy	54,186	7.9%	Mcr	Sca	Utw	Non	Lin	Mo	Lin	Raw	Rea	0
minijax	53,907	7.9%	Ful	Jav	JAX	Jty	Lin	My	Lin	Ful	Rea	0
tornado-py3	53,537	7.8%	Plt	Py	Non	Tor	Lin	Mo	Lin	Raw	Rea	0
web2py-optimized	50,933	7.5%	Ful	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
dropwizard	44,607	6.5%	Ful	Jav	JAX	Jty	Lin	Non	Lin	Ful	Rea	0
bottle-pypy2	43,726	6.4%	Mcr	Py	Tor	Non	Lin	My	Lin	Ful	Rea	0
aah	37,377	5.5%	Ful	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
comsat-servlet-under	30,636	4.5%	Plt	Jav	Svt	Non	Lin	Non	Lin	Raw	Rea	0
spring	26,114	3.8%	Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	0
web2py	20,848	3.1%	Ful	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
ffead-cpp-apache-pos	17,053	2.5%	Ful	C++	ffe	Apc	Lin	Pg	Lin	Ful	Rea	0
ffead-cpp-nginx-post	16,582	2.4%	Ful	C++	ffe	ngx	Lin	Pg	Lin	Ful	Rea	0
ffead-cpp-postgresql	16,325	2.4%	Ful	C++	ffe	ffe	Lin	Pg	Lin	Ful	Rea	0
cppcms-postgres	16,186	2.4%	Plt	C++	Non	Non	Lin	Pg	Lin	Raw	Rea	0
cppcms	15,479	2.3%	Plt	C++	Non	Non	Lin	My	Lin	Raw	Rea	0
ffead-cpp-nginx-mong	14,634	2.1%	Ful	C++	ffe	ngx	Lin	Mo	Lin	Ful	Rea	277,520
ffead-cpp-apache-mon	12,853	1.9%	Ful	C++	ffe	Apc	Lin	Mo	Lin	Ful	Rea	377,071
tornado-postgresql-r	12,705	1.9%	Plt	Py	Non	Tor	Lin	Pg	Lin	Raw	Rea	0
ffead-cpp	12,012	1.8%	Ful	C++	ffe	ffe	Lin	Mo	Lin	Ful	Rea	600,563
flask-pypy2	11,777	1.7%	Mcr	Py	Non	Tor	Lin	My	Lin	Ful	Rea	0
cppcms-direct	11,366	1.7%	Plt	C++	Non	Non	Lin	My	Lin	Raw	Rea	0
ninja-standalone	10,946	1.6%	Ful	Jav	Jty	Non	Lin	My	Lin	Ful	Rea	0
bottle-nginx-uwsgi	4,517	0.7%	Mcr	Py	uWS	ngx	Lin	My	Lin	Ful	Rea	0
flask-nginx-uwsgi	4,346	0.6%	Mcr	Py	Non	ngx	Lin	My	Lin	Ful	Rea	0
weppy-nginx-uwsgi	4,340	0.6%	Ful	Py	uWS	ngx	Lin	Pg	Lin	Ful	Rea	0
klein	822	0.1%	Mcr	Py	Non	Twi	Lin	My	Lin	Ful	Rea	0
webware	755	0.1%	Mcr	Py	Non	Non	Lin	My	Lin	Ful	Rea	0
cherrypy-py3	484	0.1%	Mcr	Py	Non	Non	Lin	My	Lin	Ful	Rea	0
cherrypy	300	0.0%	Mcr	Py	Non	Non	Lin	My	Lin	Ful	Rea	201
ffead-cpp-apache-mys	0	0.0%	Ful	C++	ffe	Apc	Lin	My	Lin	Ful	Rea	0
ffead-cpp-mysql	0	0.0%	Ful	C++	ffe	ffe	Lin	My	Lin	Ful	Rea	0
ffead-cpp-nginx-mysq	0	0.0%	Ful	C++	ffe	ngx	Lin	My	Lin	Ful	Rea	0
luna	—	Did not complete	Ful	C++	Non	Non	Lin	Non	Lin	Raw	Rea	—
luna-epoll	—	Did not complete	Ful	C++	Non	Non	Lin	Non	Lin	Raw	Rea	—
luna-thread-per-conn	—	Did not complete	Ful	C++	Non	Non	Lin	Non	Lin	Raw	Rea	—
wheezyweb-py3	—	Did not complete	Mcr	Py	Non	Mei	Lin	My	Lin	Ful	Rea	—

JSON responses per second, Dell servers at ServerCentral

Framework	16	32	64	128	256	512	Best	Cls	Lng	Plt	FE	Aos	DB	Dos	Orm	IA	Errors
aah	27,100	25,104	26,116	33,357	36,158	37,377	37,377	Ful	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
actframework	119,650	188,778	241,106	317,891	365,406	389,125	389,125	Ful	Jav	Utw	Non	Lin	Non	Lin	Ful	Rea	0
activeweb	32,552	49,290	54,223	57,299	53,239	46,884	57,299	Ful	Jav	Svt	Non	Lin	My	Lin	Mcr	Rea	0
activeweb-jackson	101,244	161,550	134,448	129,331	130,644	128,265	161,550	Ful	Jav	Svt	Non	Lin	My	Lin	Mcr	Rea	0
aiohttp	38,458	71,735	118,526	154,784	174,316	182,406	182,406	Mcr	Py	asy	Gun	Lin	Pg	Lin	Ful	Rea	0
akka-http	25,807	47,132	90,432	95,696	99,798	102,259	102,259	Mcr	Sca	Akk	Non	Lin	My	Lin	Raw	Rea	0
apistar	99,622	185,460	272,954	395,164	322,347	575,507	575,507	Mcr	Py	Non	Mei	Lin	Non	Lin	Raw	Rea	0
api_hour	24,589	45,123	79,790	93,122	99,088	100,836	100,836	Mcr	Py	asy	Gun	Lin	Pg	Lin	Raw	Rea	0
blade	108,066	162,868	213,200	302,773	327,933	324,139	327,933	Ful	Jav	Non	Non	Lin	My	Lin	Ful	Rea	0
blaze	131,135	222,745	301,708	460,695	586,170	617,537	617,537	Mcr	Sca	bla	Non	Lin	Non	Lin	Raw	Rea	0
bottle	75,095	137,640	208,593	253,584	350,609	364,428	364,428	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
bottle-nginx-uwsgi	4,282	4,410	4,517	4,447	4,232	4,167	4,517	Mcr	Py	uWS	ngx	Lin	My	Lin	Ful	Rea	0
bottle-py3	64,303	124,305	199,251	252,690	293,097	298,316	298,316	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
bottle-pypy2	20,845	36,299	40,539	30,470	43,330	43,726	43,726	Mcr	Py	Tor	Non	Lin	My	Lin	Ful	Rea	0
colossus	124,403	199,263	310,918	305,589	313,324	497,039	497,039	Mcr	Sca	Akk	Non	Lin	Non	Lin	Raw	Rea	0
cpoll_cppsp	157,112	264,756	386,963	241,145	646,776	608,473	646,776	Plt	C++	Non	Non	Lin	Non	Lin	Raw	Rea	0
curacao	48,236	95,375	109,121	107,083	107,178	109,172	109,172	Mcr	Jav	Svt	Non	Lin	Non	Lin	Raw	Rea	5
cutelyst	16,170	22,148	36,966	50,368	70,352	93,025	93,025	Ful	C++	Qt	Non	Lin	Non	Lin	Raw	Rea	0
cutelyst-nginx	35,055	67,550	98,740	86,141	84,374	83,521	98,740	Ful	C++	Qt	ngx	Lin	Non	Lin	Raw	Rea	0
cutelyst-pf-epoll	22,329	25,216	15,670	93,555	90,251	74,450	93,555	Ful	C++	Qt	Non	Lin	Non	Lin	Raw	Rea	0
cutelyst-thread	104,103	188,941	281,529	405,116	313,558	611,406	611,406	Ful	C++	Qt	Non	Lin	Non	Lin	Raw	Rea	0
cutelyst-thread-epol	109,589	202,258	313,176	220,329	559,997	220,257	559,997	Ful	C++	Qt	Non	Lin	Non	Lin	Raw	Rea	0
cutelyst-thread-epol	108,972	186,039	283,308	333,249	461,957	220,518	461,957	Ful	C++	Qt	Non	Lin	Non	Lin	Raw	Rea	0
django	22,718	41,959	70,542	77,278	74,973	71,129	77,278	Ful	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
django-py3	36,802	68,452	115,326	137,821	146,354	145,614	146,354	Ful	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
dropwizard	34,450	40,015	42,826	43,505	43,911	44,607	44,607	Ful	Jav	JAX	Jty	Lin	Non	Lin	Ful	Rea	0
falcon	79,436	155,253	249,605	224,089	344,728	477,082	477,082	Mcr	Py	Non	Mei	Lin	Non	Lin	Raw	Rea	0
falcon-py3	80,510	149,309	235,279	311,946	353,057	424,771	424,771	Mcr	Py	Non	Mei	Lin	Non	Lin	Raw	Rea	0
falcon-pypy2	41,931	70,953	97,814	131,950	179,668	199,972	199,972	Mcr	Py	Non	Tor	Lin	Non	Lin	Raw	Rea	0
falcore	89,877	138,785	155,612	213,491	224,361	229,102	229,102	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
ffead-cpp	12,012	0	0	0	0	0	12,012	Ful	C++	ffe	ffe	Lin	Mo	Lin	Ful	Rea	600,563
ffead-cpp-apache-mon	12,853	0	0	0	0	0	12,853	Ful	C++	ffe	Apc	Lin	Mo	Lin	Ful	Rea	377,071
ffead-cpp-apache-mys	0	0	0	0	0	0	0	Ful	C++	ffe	Apc	Lin	My	Lin	Ful	Rea	0
ffead-cpp-apache-pos	16,815	17,053	16,907	16,602	16,259	16,051	17,053	Ful	C++	ffe	Apc	Lin	Pg	Lin	Ful	Rea	0
ffead-cpp-mysql	0	0	0	0	0	0	0	Ful	C++	ffe	ffe	Lin	My	Lin	Ful	Rea	0
ffead-cpp-nginx-mong	14,634	0	0	0	0	0	14,634	Ful	C++	ffe	ngx	Lin	Mo	Lin	Ful	Rea	277,520
ffead-cpp-nginx-mysq	0	0	0	0	0	0	0	Ful	C++	ffe	ngx	Lin	My	Lin	Ful	Rea	0
ffead-cpp-nginx-post	16,582	16,245	16,021	16,303	16,196	15,961	16,582	Ful	C++	ffe	ngx	Lin	Pg	Lin	Ful	Rea	0
ffead-cpp-postgresql	16,325	0	0	0	0	0	16,325	Ful	C++	ffe	ffe	Lin	Pg	Lin	Ful	Rea	0
finagle	96,193	169,883	254,018	311,887	370,170	420,556	420,556	Mcr	Sca	Nty	Non	Lin	Non	Lin	Raw	Rea	0
finatra	100,474	185,679	301,036	388,764	462,553	523,712	523,712	Mcr	Sca	Nty	Non	Lin	Non	Lin	Raw	Rea	0
finch	93,950	161,438	262,923	332,795	375,221	396,796	396,796	Mcr	Sca	Nty	Non	Lin	Non	Lin	Raw	Rea	0
fintrospect	90,821	168,566	278,878	372,977	433,910	504,706	504,706	Mcr	Sca	Nty	Non	Lin	My	Lin	Raw	Rea	0
flask	22,513	44,516	71,819	80,332	83,230	80,731	83,230	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
flask-nginx-uwsgi	4,346	4,192	4,236	4,209	4,266	4,237	4,346	Mcr	Py	Non	ngx	Lin	My	Lin	Ful	Rea	0
flask-py3	23,232	43,210	79,350	93,163	93,918	92,057	93,918	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
flask-pypy2	6,239	10,826	11,777	11,270	7,829	7,194	11,777	Mcr	Py	Non	Tor	Lin	My	Lin	Ful	Rea	0
gemini	139,945	220,579	279,808	213,162	498,666	360,890	498,666	Ful	Jav	Svt	Res	Lin	Non	Lin	Mcr	Rea	0
gin	58,846	95,670	148,032	213,352	237,096	282,655	282,655	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
goji	53,456	82,014	48,578	24,192	23,656	23,851	82,014	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
grizzly	126,373	222,082	337,596	273,515	587,363	641,988	641,988	Mcr	Jav	Svt	Non	Lin	Non	Lin	Raw	Rea	0
grizzly-jersey	67,143	119,970	155,625	206,787	232,048	286,502	286,502	Mcr	Jav	JAX	Non	Lin	My	Lin	Ful	Rea	0
http4s	47,825	82,059	82,652	80,886	80,425	81,808	82,652	Mcr	Sca	NIO	bla	Lin	Pg	Lin	Raw	Rea	0
japronto	114,871	207,914	274,505	223,871	374,369	629,566	629,566	Mcr	Py	Non	Non	Lin	Non	Lin	Raw	Rea	0
jawn	53,480	52,414	125,468	138,805	159,717	162,837	162,837	Ful	Jav	Svt	Utw	Lin	Pg	Lin	Raw	Rea	0
undertow-jersey-c3p0	50,850	53,966	116,212	140,923	144,038	145,981	145,981	Plt	Jav	JAX	Non	Lin	My	Lin	Ful	Rea	0
undertow-jersey-hika	48,848	51,366	103,878	142,545	146,497	145,140	146,497	Plt	Jav	JAX	Non	Lin	My	Lin	Ful	Rea	0
jooby	74,001	86,020	93,221	96,390	101,330	99,181	101,330	Ful	Jav	Nty	Non	Lin	Pg	Lin	Raw	Rea	0
kami	60,720	99,972	170,014	251,574	268,516	309,291	309,291	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
klein	822	816	820	819	807	796	822	Mcr	Py	Non	Twi	Lin	My	Lin	Ful	Rea	0
luna	— Did not complete —							Ful	C++	Non	Non	Lin	Non	Lin	Raw	Rea	—
luna-epoll	— Did not complete —							Ful	C++	Non	Non	Lin	Non	Lin	Raw	Rea	—
luna-thread-per-conn	— Did not complete —							Ful	C++	Non	Non	Lin	Non	Lin	Raw	Rea	—
minijax	46,021	52,217	53,316	53,200	53,907	52,841	53,907	Ful	Jav	JAX	Jty	Lin	My	Lin	Ful	Rea	0
morepath	48,181	94,207	162,028	196,807	219,172	218,625	219,172	Mcr	Py	Mei	Gun	Lin	Pg	Lin	Ful	Rea	0
netty	155,942	259,002	357,255	219,667	590,560	589,815	590,560	Plt	Jav	Nty	Non	Lin	Non	Lin	Raw	Rea	0
ninja-standalone	9,145	8,943	8,977	9,213	10,837	10,946	10,946	Ful	Jav	Jty	Non	Lin	My	Lin	Ful	Rea	0
bayou	127,662	139,946	124,698	122,035	132,323	152,958	152,958	Plt	Jav	Bay	Non	Lin	Non	Lin	Raw	Rea	0
beego	55,457	92,391	133,949	201,432	226,290	271,747	271,747	Ful	Go	Non	Non	Lin	Non	Lin	Raw	Rea	0
cherrypy	291	290	288	290	300	299	300	Mcr	Py	Non	Non	Lin	My	Lin	Ful	Rea	201
cherrypy-py3	484	481	469	467	468	469	484	Mcr	Py	Non	Non	Lin	My	Lin	Ful	Rea	0
comsat-servlet	32,491	42,259	36,635	33,425	55,583	36,282	55,583	Plt	Jav	Svt	Non	Lin	Non	Lin	Raw	Rea	0
comsat-servlet-under	30,636	23,921	17,431	18,573	16,120	15,660	30,636	Plt	Jav	Svt	Non	Lin	Non	Lin	Raw	Rea	0
comsat-webactors	47,242	74,246	53,812	34,884	72,329	48,794	74,246	Plt	Jav	Com	Non	Lin	Non	Lin	Raw	Rea	0
comsat-webactors-und	54,155	83,409	59,228	97,801	56,729	76,236	97,801	Plt	Jav	Com	Non	Lin	Non	Lin	Raw	Rea	0
cppcms	15,479	15,142	15,161	15,213	15,209	15,209	15,479	Plt	C++	Non	Non	Lin	My	Lin	Raw	Rea	0
cppcms-direct	11,366	11,092	11,226	11,262	11,272	11,271	11,366	Plt	C++	Non	Non	Lin	My	Lin	Raw	Rea	0
cppcms-postgres	16,186	15,945	15,963	15,603	15,754	15,914	16,186	Plt	C++	Non	Non	Lin	Pg	Lin	Raw	Rea	0
echo	58,785	85,735	125,455	138,014	138,023	139,626	139,626	Mcr	Go	Non	Non	Lin	Pg	Lin	Raw	Rea	0
fasthttp	129,812	148,221	174,704	258,719	310,145	357,675	357,675	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
go	61,808	98,620	153,752	222,346	248,717	296,356	296,356	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
go-prefork	109,268	188,295	284,100	243,081	436,384	603,592	603,592	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
jetty	61,200	99,346	163,217	177,118	166,748	161,382	177,118	Plt	Jav	Jty	Non	Lin	Non	Lin	Raw	Rea	0
jetty-servlet	57,848	95,857	167,820	182,147	179,944	177,650	182,147	Plt	Jav	Jty	Non	Lin	Non	Lin	Raw	Rea	0
jlhttp	151,273	246,542	360,176	323,820	315,819	311,241	360,176	Plt	Jav	JLH	Non	Lin	Non	Lin	Raw	Rea	0
light-4j	137,299	232,593	352,417	276,551	634,889	634,167	634,889	Plt	Jav	Lig	Non	Lin	Non	Lin	Raw	Rea	0
revel	49,396	84,421	112,802	109,833	99,115	102,555	112,802	Ful	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
servlet	163,221	250,186	308,237	500,631	354,718	657,791	657,791	Plt	Jav	Svt	Res	Lin	Non	Lin	Raw	Rea	0
servlet-afterburner	164,924	260,427	354,688	217,628	434,911	662,134	662,134	Plt	Jav	Svt	Res	Lin	Non	Lin	Raw	Rea	0
servlet-cjs	164,149	245,857	267,987	350,563	626,197	663,623	663,623	Plt	Jav	Svt	Res	Lin	Non	Lin	Raw	Rea	0
tornado	15,719	29,785	49,955	59,293	59,224	59,088	59,293	Plt	Py	Non	Tor	Lin	Mo	Lin	Raw	Rea	0
tornado-postgresql-r	12,570	12,594	12,705	12,598	12,516	12,465	12,705	Plt	Py	Non	Tor	Lin	Pg	Lin	Raw	Rea	0
tornado-py3	13,647	26,007	43,153	53,537	52,977	52,951	53,537	Plt	Py	Non	Tor	Lin	Mo	Lin	Raw	Rea	0
tornado-pypy2	53,027	93,975	144,063	193,115	196,384	195,033	196,384	Plt	Py	Non	Tor	Lin	Mo	Lin	Raw	Rea	0
undertow	145,048	226,868	271,403	491,805	354,520	595,431	595,431	Plt	Jav	Utw	Non	Lin	Non	Lin	Raw	Rea	0
vertx	137,383	229,785	331,541	445,892	513,059	550,381	550,381	Plt	Jav	Ver	Non	Lin	Non	Lin	Raw	Rea	0
wildfly-ee7	32,858	30,260	53,696	65,508	62,588	70,103	70,103	Ful	Jav	Svt	Wil	Lin	My	Lin	Ful	Rea	0
play1	22,373	31,957	44,117	55,266	52,679	51,836	55,266	Ful	Jav	Nty	Non	Lin	My	Lin	Ful	Rea	0
play2	36,164	38,861	50,770	57,184	58,043	56,849	58,043	Ful	Jav	Akk	Non	Lin	Non	Lin	Raw	Rea	0
play2-java-netty	57,518	95,693	129,018	171,522	176,509	174,205	176,509	Ful	Jav	Nty	Non	Lin	Non	Lin	Raw	Rea	0
play2-scala	53,957	80,542	93,002	91,751	90,167	88,810	93,002	Ful	Sca	Akk	Non	Lin	Non	Lin	Raw	Rea	0
play2-scala-scala-ne	68,860	119,708	191,512	208,400	185,559	108,423	208,400	Ful	Sca	Nty	Non	Lin	Non	Lin	Raw	Rea	0
proteus	130,635	218,428	311,447	343,015	592,389	658,410	658,410	Mcr	Jav	Utw	Non	Lin	Pg	Lin	Raw	Rea	0
pyramid	36,853	72,815	117,230	146,094	157,668	157,687	157,687	Ful	Py	Non	Mei	Lin	Pg	Lin	Ful	Rea	0
pyramid-py2	41,477	79,526	135,451	161,736	174,835	175,201	175,201	Ful	Py	Non	Mei	Lin	Pg	Lin	Ful	Rea	0
rapidoid	154,597	257,741	303,439	435,543	229,629	662,269	662,269	Plt	Jav	Rap	Non	Lin	Non	Lin	Raw	Rea	0
rapidoid-http-fast	158,925	186,982	334,638	225,098	633,509	642,181	642,181	Plt	Jav	Rap	Non	Lin	Non	Lin	Raw	Rea	0
restexpress	32,108	53,759	115,937	115,157	115,250	114,522	115,937	Mcr	Jav	Nty	Non	Lin	Mo	Lin	Raw	Rea	0
revenj-jvm	166,623	247,393	380,956	239,216	569,099	669,526	669,526	Ful	Jav	Svt	Res	Lin	Pg	Lin	Ful	Rea	0
s-server	148,541	230,120	206,888	360,200	633,811	561,419	633,811	Plt	Sca	s-s	Non	Lin	Non	Lin	Raw	Rea	0
sanic	43,208	97,073	154,921	205,390	204,900	219,254	219,254	Mcr	Py	Non	Non	Lin	Non	Lin	Raw	Rea	0
scruffy	23,019	31,896	43,847	54,186	48,092	46,542	54,186	Mcr	Sca	Utw	Non	Lin	Mo	Lin	Raw	Rea	0
silicon	124,680	156,779	133,132	134,027	134,691	133,617	156,779	Mcr	C++	Non	mic	Lin	My	Lin	Ful	Rea	0
spark	75,127	132,965	180,744	225,691	203,579	200,415	225,691	Mcr	Jav	Svt	Res	Lin	My	Lin	Ful	Rea	0
spray	34,929	107,382	108,327	131,403	139,686	148,368	148,368	Mcr	Sca	Akk	Non	Lin	Non	Lin	Raw	Rea	0
spring	26,114	25,525	24,419	23,176	23,365	23,502	26,114	Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	0
tapestry	67,441	119,881	146,931	135,610	133,409	127,889	146,931	Ful	Jav	Svt	Res	Lin	My	Lin	Ful	Rea	0
treefrog	79,063	148,879	238,267	304,265	219,226	221,419	304,265	Ful	C++	Non	Non	Lin	My	Lin	Mcr	Rea	0
treefrog-hybrid	21,229	32,191	43,403	56,445	65,807	69,681	69,681	Ful	C++	Non	Non	Lin	My	Lin	Mcr	Rea	18
treefrog-mongodb	78,362	149,258	251,523	306,788	233,016	243,818	306,788	Ful	C++	Non	Non	Lin	Mo	Lin	Mcr	Rea	0
treefrog-postgres	78,448	147,887	246,267	293,190	256,231	276,258	293,190	Ful	C++	Non	Non	Lin	Pg	Lin	Mcr	Rea	0
turbogears	24,318	40,339	79,672	86,535	79,574	77,014	86,535	Mcr	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
ulib-json	175,758	286,111	363,723	284,583	618,440	682,383	682,383	Plt	C++	Non	ULi	Lin	Non	Lin	Mcr	Rea	0
ulib-json_fit	182,215	291,335	251,618	557,083	265,877	664,458	664,458	Plt	C++	Non	ULi	Lin	Non	Lin	Mcr	Rea	0
uvicorn	80,604	153,377	242,204	329,965	313,032	475,636	475,636	Plt	Py	Non	Non	Lin	Pg	Lin	Raw	Rea	0
vertx-web	126,694	221,640	331,125	289,053	548,259	570,243	570,243	Mcr	Jav	vtx	Non	Lin	Non	Lin	Raw	Rea	0
web2py	7,356	13,976	20,834	20,848	19,805	19,836	20,848	Ful	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
web2py-optimized	16,192	31,342	47,346	50,933	47,024	46,217	50,933	Ful	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
webgo	53,917	96,254	131,116	139,313	136,670	137,543	139,313	Mcr	Go	Non	Non	Lin	Non	Lin	Raw	Rea	0
webware	755	743	737	734	740	734	755	Mcr	Py	Non	Non	Lin	My	Lin	Ful	Rea	0
weppy	52,438	106,094	176,706	210,161	249,253	211,690	249,253	Ful	Py	Mei	Non	Lin	Pg	Lin	Ful	Rea	0
weppy-nginx-uwsgi	4,141	4,204	4,340	4,326	4,164	4,101	4,340	Ful	Py	uWS	ngx	Lin	Pg	Lin	Ful	Rea	0
weppy-py3	52,462	96,732	172,127	215,281	158,143	218,228	218,228	Ful	Py	Mei	Non	Lin	Pg	Lin	Ful	Rea	0
weppy-pypy2	15,405	35,915	54,640	50,124	58,878	62,790	62,790	Ful	Py	Tor	Non	Lin	Pg	Lin	Ful	Rea	0
wheezyweb	79,557	145,573	235,502	301,839	380,972	387,714	387,714	Mcr	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
wheezyweb-py3	— Did not complete —							Mcr	Py	Non	Mei	Lin	My	Lin	Ful	Rea	—
wicket	104,203	168,138	228,476	265,298	283,711	268,136	283,711	Ful	Jav	Svt	Res	Lin	My	Lin	Ful	Rea	0
wt	13,419	13,507	38,638	59,385	58,066	57,717	59,385	Ful	C++	Non	Non	Lin	My	Lin	Ful	Rea	0

Latency of JSON responses, Dell servers at ServerCentral

Framework	Average latency (lower is better)		σ (SD)	Max	Errors
cutelyst-thread	0.8 ms	0.1%	0.6 ms	28.7 ms	0
ulib-json_fit	0.9 ms	0.1%	0.9 ms	25.1 ms	0
cpoll_cppsp	0.9 ms	0.1%	0.8 ms	17.8 ms	0
ulib-json	0.9 ms	0.1%	1.3 ms	29.6 ms	0
s-server	1.1 ms	0.1%	1.7 ms	44.9 ms	0
light-4j	1.2 ms	0.1%	2.7 ms	87.9 ms	0
proteus	1.2 ms	0.1%	4.3 ms	206.3 ms	0
colossus	1.2 ms	0.1%	1.7 ms	44.1 ms	0
rapidoid-http-fast	1.4 ms	0.1%	7.2 ms	203.5 ms	0
vertx	1.4 ms	0.1%	2.4 ms	48.4 ms	0
undertow	1.4 ms	0.1%	5.8 ms	279.6 ms	0
revenj-jvm	1.4 ms	0.1%	8.2 ms	212.1 ms	0
servlet-afterburner	1.4 ms	0.1%	6.4 ms	204.5 ms	0
netty	1.5 ms	0.1%	6.5 ms	209.0 ms	0
grizzly	1.5 ms	0.1%	8.7 ms	232.0 ms	0
japronto	1.5 ms	0.1%	10.4 ms	410.1 ms	0
servlet-cjs	1.5 ms	0.1%	8.2 ms	212.4 ms	0
servlet	1.5 ms	0.1%	7.8 ms	210.7 ms	0
go-prefork	1.6 ms	0.1%	9.2 ms	212.0 ms	0
rapidoid	1.6 ms	0.1%	9.4 ms	212.9 ms	0
apistar	1.6 ms	0.1%	9.9 ms	211.5 ms	0
fasthttp	1.7 ms	0.2%	7.0 ms	227.3 ms	0
uvicorn	1.7 ms	0.2%	8.9 ms	230.1 ms	0
falcon	1.8 ms	0.2%	9.1 ms	217.3 ms	0
kami	1.9 ms	0.2%	2.1 ms	55.3 ms	0
falcon-py3	2.1 ms	0.2%	9.9 ms	212.3 ms	0
go	2.2 ms	0.2%	2.6 ms	45.3 ms	0
cutelyst-thread-epol	2.3 ms	0.2%	0.9 ms	10.7 ms	0
cutelyst-thread-epol	2.3 ms	0.2%	1.1 ms	25.2 ms	0
beego	2.5 ms	0.2%	3.0 ms	54.8 ms	0
gin	2.6 ms	0.2%	7.1 ms	212.9 ms	0
vertx-web	2.8 ms	0.3%	12.6 ms	259.7 ms	0
blade	3.0 ms	0.3%	4.2 ms	65.2 ms	0
weppy-py3	3.1 ms	0.3%	8.5 ms	222.1 ms	0
jawn	3.4 ms	0.3%	3.6 ms	205.8 ms	0
wheezyweb	3.4 ms	0.3%	11.6 ms	376.9 ms	0
aiohttp	3.5 ms	0.3%	10.5 ms	221.3 ms	0
sanic	3.6 ms	0.3%	13.7 ms	423.2 ms	0
echo	3.7 ms	0.3%	1.6 ms	40.4 ms	0
weppy	3.8 ms	0.3%	10.8 ms	323.2 ms	0
actframework	3.9 ms	0.3%	8.4 ms	209.7 ms	0
bottle	4.2 ms	0.4%	12.1 ms	430.2 ms	0
bottle-py3	4.4 ms	0.4%	11.2 ms	359.8 ms	0
falcon-pypy2	4.4 ms	0.4%	14.5 ms	397.8 ms	0
tornado-pypy2	4.5 ms	0.4%	14.5 ms	447.0 ms	0
undertow-jersey-c3p0	4.5 ms	0.4%	9.0 ms	245.8 ms	0
treefrog-postgres	4.6 ms	0.4%	31.2 ms	837.9 ms	0
falcore	4.8 ms	0.4%	8.8 ms	246.8 ms	0
spray	5.1 ms	0.5%	17.1 ms	426.7 ms	0
treefrog	5.1 ms	0.5%	27.4 ms	814.9 ms	0
undertow-jersey-hika	5.2 ms	0.5%	12.5 ms	305.1 ms	0
jlhttp	5.3 ms	0.5%	11.9 ms	409.3 ms	0
cutelyst	5.6 ms	0.5%	2.4 ms	26.6 ms	0
morepath	5.8 ms	0.5%	13.5 ms	477.3 ms	0
api_hour	6.0 ms	0.5%	12.5 ms	414.3 ms	0
finch	6.1 ms	0.5%	25.6 ms	828.6 ms	0
jooby	6.1 ms	0.5%	7.6 ms	119.7 ms	0
webgo	6.4 ms	0.6%	8.4 ms	233.0 ms	0
blaze	6.4 ms	0.6%	56.0 ms	1600.0 ms	0
spark	7.3 ms	0.7%	12.0 ms	223.4 ms	0
pyramid	7.5 ms	0.7%	16.5 ms	435.2 ms	0
revel	7.7 ms	0.7%	10.5 ms	262.5 ms	0
cutelyst-nginx	7.8 ms	0.7%	8.2 ms	121.1 ms	0
activeweb-jackson	7.8 ms	0.7%	11.8 ms	222.9 ms	0
fintrospect	8.1 ms	0.7%	66.0 ms	1640.0 ms	0
finatra	8.3 ms	0.7%	39.9 ms	862.3 ms	0
curacao	8.3 ms	0.7%	18.8 ms	520.8 ms	5
bayou	8.5 ms	0.8%	51.8 ms	1610.0 ms	0
cutelyst-pf-epoll	8.9 ms	0.8%	10.1 ms	152.2 ms	0
pyramid-py2	9.3 ms	0.8%	20.6 ms	533.3 ms	0
tornado	9.3 ms	0.8%	12.0 ms	379.2 ms	0
treefrog-mongodb	9.7 ms	0.9%	71.2 ms	1620.0 ms	0
tornado-py3	9.9 ms	0.9%	10.3 ms	410.5 ms	0
flask-py3	9.9 ms	0.9%	19.7 ms	638.9 ms	0
django-py3	10.3 ms	0.9%	23.9 ms	717.9 ms	0
finagle	10.4 ms	0.9%	66.5 ms	1690.0 ms	0
play2-java-netty	10.7 ms	1.0%	31.3 ms	512.2 ms	0
restexpress	10.9 ms	1.0%	44.3 ms	1040.0 ms	0
grizzly-jersey	11.5 ms	1.0%	40.1 ms	476.6 ms	0
dropwizard	11.5 ms	1.0%	3.7 ms	81.2 ms	0
jetty-servlet	11.5 ms	1.0%	76.6 ms	1640.0 ms	0
treefrog-hybrid	11.8 ms	1.1%	55.6 ms	1640.0 ms	18
gemini	11.9 ms	1.1%	75.1 ms	910.4 ms	0
wicket	12.0 ms	1.1%	24.3 ms	343.1 ms	0
flask	13.3 ms	1.2%	23.7 ms	478.0 ms	0
silicon	14.0 ms	1.2%	96.1 ms	1670.0 ms	0
turbogears	14.0 ms	1.3%	26.3 ms	689.0 ms	0
jetty	14.3 ms	1.3%	69.0 ms	1710.0 ms	0
wt	14.3 ms	1.3%	48.7 ms	933.5 ms	0
django	15.0 ms	1.3%	26.5 ms	616.2 ms	0
comsat-webactors-und	16.5 ms	1.5%	68.9 ms	635.8 ms	0
minijax	16.7 ms	1.5%	71.8 ms	1650.0 ms	0
wildfly-ee7	17.3 ms	1.5%	28.8 ms	415.3 ms	0
play1	17.6 ms	1.6%	81.5 ms	1610.0 ms	0
http4s	18.1 ms	1.6%	102.2 ms	1600.0 ms	0
web2py-optimized	18.1 ms	1.6%	26.2 ms	428.2 ms	0
cherrypy-py3	21.3 ms	1.9%	5.1 ms	233.8 ms	0
goji	22.5 ms	2.0%	18.0 ms	129.7 ms	0
play2-scala	22.8 ms	2.0%	89.3 ms	1470.0 ms	0
spring	24.4 ms	2.2%	19.3 ms	230.7 ms	0
activeweb	24.6 ms	2.2%	44.0 ms	568.0 ms	0
tapestry	24.9 ms	2.2%	40.3 ms	428.7 ms	0
ffead-cpp-nginx-post	31.5 ms	2.8%	2.8 ms	57.5 ms	0
ffead-cpp-apache-pos	31.6 ms	2.8%	2.6 ms	60.2 ms	0
cppcms-postgres	32.2 ms	2.9%	3.3 ms	41.2 ms	0
aah	33.1 ms	3.0%	73.7 ms	1080.0 ms	0
cherrypy	33.3 ms	3.0%	7.1 ms	253.0 ms	201
weppy-pypy2	33.3 ms	3.0%	76.3 ms	1610.0 ms	0
cppcms	33.6 ms	3.0%	2.5 ms	40.5 ms	0
play2-scala-scala-ne	34.0 ms	3.0%	56.4 ms	595.4 ms	0
bottle-pypy2	34.8 ms	3.1%	74.8 ms	1360.0 ms	0
akka-http	36.3 ms	3.2%	118.0 ms	1300.0 ms	0
web2py	38.9 ms	3.5%	53.4 ms	798.7 ms	0
tornado-postgresql-r	39.5 ms	3.5%	9.0 ms	235.4 ms	0
cppcms-direct	44.2 ms	3.9%	4.6 ms	247.0 ms	0
play2	52.2 ms	4.7%	248.6 ms	3300.0 ms	0
ninja-standalone	54.6 ms	4.9%	97.3 ms	1700.0 ms	0
flask-pypy2	95.9 ms	8.6%	119.4 ms	1390.0 ms	0
scruffy	108.6 ms	9.7%	452.1 ms	4890.0 ms	0
flask-nginx-uwsgi	120.3 ms	10.7%	14.3 ms	361.3 ms	0
bottle-nginx-uwsgi	123.0 ms	11.0%	14.3 ms	361.9 ms	0
comsat-servlet-under	124.9 ms	11.1%	203.7 ms	1170.0 ms	0
weppy-nginx-uwsgi	125.1 ms	11.2%	15.0 ms	387.7 ms	0
comsat-webactors	545.9 ms	48.7%	1030.0 ms	4430.0 ms	0
klein	599.2 ms	53.5%	256.0 ms	2890.0 ms	0
webware	681.4 ms	60.8%	82.1 ms	734.5 ms	0
comsat-servlet	1120.0 ms	100.0%	1960.0 ms	7580.0 ms	0

Framework overhead of JSON responses, Dell servers at ServerCentral

Framework best		vs	Platform best		Ratio (higher is better)
tornado-pypy2	196,384	vs	tornado	59,293	331.2%
go-prefork	603,592	vs	go	296,356	203.7%
finatra	523,712	vs	finagle	420,556	124.5%
fasthttp	357,675	vs	go	296,356	120.7%
fintrospect	504,706	vs	finagle	420,556	120.0%
kami	309,291	vs	go	296,356	104.4%
vertx-web	570,243	vs	vertx	550,381	103.6%
revenj-jvm	669,526	vs	servlet	657,791	101.8%
servlet-cjs	663,623	vs	servlet	657,791	100.9%
treefrog-mongodb	306,788	vs	treefrog	304,265	100.8%
servlet-afterburner	662,134	vs	servlet	657,791	100.7%
undertow-jersey-hika	146,497	vs	undertow-jersey-c3p0	145,981	100.4%
undertow-jersey-c3p0	145,981	vs	undertow-jersey-hika	146,497	99.6%
gin	282,655	vs	go	296,356	95.4%
finch	396,796	vs	finagle	420,556	94.4%
beego	271,747	vs	go	296,356	91.7%
play2-scala	93,002	vs	akka-http	102,259	90.9%
tornado-py3	53,537	vs	tornado	59,293	90.3%
falcon	477,082	vs	wsgi	604,690	78.9%
falcore	229,102	vs	go	296,356	77.3%
gemini	498,666	vs	servlet	657,791	75.8%
finagle	420,556	vs	netty	590,560	71.2%
falcon-py3	424,771	vs	wsgi	604,690	70.2%
actframework	389,125	vs	undertow	595,431	65.4%
bottle	364,428	vs	wsgi	604,690	60.3%
play2	58,043	vs	akka-http	102,259	56.8%
bottle-py3	298,316	vs	wsgi	604,690	49.3%
echo	139,626	vs	go	296,356	47.1%
webgo	139,313	vs	go	296,356	47.0%
wicket	283,711	vs	servlet	657,791	43.1%
weppy	249,253	vs	wsgi	604,690	41.2%
revel	112,802	vs	go	296,356	38.1%
weppy-py3	218,228	vs	wsgi	604,690	36.1%
play2-scala-scala-ne	208,400	vs	netty	590,560	35.3%
spark	225,691	vs	servlet	657,791	34.3%
falcon-pypy2	199,972	vs	wsgi	604,690	33.1%
play2-java-netty	176,509	vs	netty	590,560	29.9%
pyramid-py2	175,201	vs	wsgi	604,690	29.0%
jetty-servlet	182,147	vs	servlet	657,791	27.7%
goji	82,014	vs	go	296,356	27.7%
jetty	177,118	vs	servlet	657,791	26.9%
pyramid	157,687	vs	wsgi	604,690	26.1%
jawn	162,837	vs	servlet	657,791	24.8%
activeweb-jackson	161,550	vs	servlet	657,791	24.6%
django-py3	146,354	vs	wsgi	604,690	24.2%
treefrog-hybrid	69,681	vs	treefrog	304,265	22.9%
tapestry	146,931	vs	servlet	657,791	22.3%
tornado-postgresql-r	12,705	vs	tornado	59,293	21.4%
restexpress	115,937	vs	netty	590,560	19.6%
jooby	101,330	vs	netty	590,560	17.2%
curacao	109,172	vs	servlet	657,791	16.6%
flask-py3	93,918	vs	wsgi	604,690	15.5%
flask	83,230	vs	wsgi	604,690	13.8%
django	77,278	vs	wsgi	604,690	12.8%
aah	37,377	vs	go	296,356	12.6%
weppy-pypy2	62,790	vs	wsgi	604,690	10.4%
play1	55,266	vs	netty	590,560	9.4%
scruffy	54,186	vs	undertow	595,431	9.1%
activeweb	57,299	vs	servlet	657,791	8.7%
web2py-optimized	50,933	vs	wsgi	604,690	8.4%
bottle-pypy2	43,726	vs	wsgi	604,690	7.2%
dropwizard	44,607	vs	servlet	657,791	6.8%
ninja-standalone	10,946	vs	jetty	177,118	6.2%
spring	26,114	vs	servlet	657,791	4.0%
web2py	20,848	vs	wsgi	604,690	3.4%
flask-pypy2	11,777	vs	wsgi	604,690	1.9%
bottle-nginx-uwsgi	4,517	vs	wsgi	604,690	0.7%
flask-nginx-uwsgi	4,346	vs	wsgi	604,690	0.7%
weppy-nginx-uwsgi	4,340	vs	wsgi	604,690	0.7%

Best (bar chart)

Data table

Latency

Framework overhead

Best (bar chart)

Data table

Latency

Framework overhead

Requirements summary

In this test, each response is a JSON serialization of a freshly-instantiated object that maps the key message to the value Hello, World!

Example response:

HTTP/1.1 200 OK Content-Type: application/json Content-Length: 28 Server: Example Date: Wed, 17 Apr 2013 12:00:00 GMT {"message":"Hello, World!"}

For a more detailed description of the requirements, see the Source Code and Requirements section.

Best plaintext responses per second, Dell servers at ServerCentral(134 tests)

Framework	Best performance (higher is better)		Cls	Lng	Plt	FE	Aos	DB	Dos	Orm	IA	Errors
ulib-plaintext_fit	2,684,333	100.0%(98.0%)	Plt	C++	Non	ULi	Lin	Non	Lin	Mcr	Rea	2
ulib	2,679,274	99.8%(97.8%)	Plt	C++	Non	ULi	Lin	Non	Lin	Mcr	Rea	2
fasthttp	2,665,278	99.3%(97.3%)	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
rapidoid-http-fast	2,654,663	98.9%(96.9%)	Plt	Jav	Rap	Non	Lin	Non	Lin	Raw	Rea	0
rapidoid	2,608,986	97.2%(95.3%)	Plt	Jav	Rap	Non	Lin	Non	Lin	Raw	Rea	0
undertow	2,544,955	94.8%(92.9%)	Plt	Jav	Utw	Non	Lin	Non	Lin	Raw	Rea	0
vertx	2,503,741	93.3%(91.4%)	Plt	Jav	Ver	Non	Lin	Non	Lin	Raw	Rea	0
netty	2,447,315	91.2%(89.4%)	Plt	Jav	Nty	Non	Lin	Non	Lin	Raw	Rea	1
proteus	2,441,688	91.0%(89.2%)	Mcr	Jav	Utw	Non	Lin	Pg	Lin	Raw	Rea	0
light-4j	2,435,150	90.7%(88.9%)	Plt	Jav	Lig	Non	Lin	Non	Lin	Raw	Rea	0
actframework	2,424,671	90.3%(88.5%)	Ful	Jav	Utw	Non	Lin	Non	Lin	Ful	Rea	0
vertx-web	2,411,118	89.8%(88.0%)	Mcr	Jav	vtx	Non	Lin	Non	Lin	Raw	Rea	0
japronto	2,356,964	87.8%(86.1%)	Mcr	Py	Non	Non	Lin	Non	Lin	Raw	Rea	178
cpoll_cppsp	2,119,565	79.0%(77.4%)	Plt	C++	Non	Non	Lin	Non	Lin	Raw	Rea	0
bayou	1,752,492	65.3%(64.0%)	Plt	Jav	Bay	Non	Lin	Non	Lin	Raw	Rea	4,674
cutelyst-thread-epol	1,625,125	60.5%(59.3%)	Ful	C++	Qt	Non	Lin	Non	Lin	Raw	Rea	0
s-server	1,560,315	58.1%(57.0%)	Plt	Sca	s-s	Non	Lin	Non	Lin	Raw	Rea	0
colossus	1,440,236	53.7%(52.6%)	Mcr	Sca	Akk	Non	Lin	Non	Lin	Raw	Rea	0
blaze	1,362,644	50.8%(49.8%)	Mcr	Sca	bla	Non	Lin	Non	Lin	Raw	Rea	2,638
cutelyst-thread	1,032,448	38.5%(37.7%)	Ful	C++	Qt	Non	Lin	Non	Lin	Raw	Rea	0
go-prefork	1,011,589	37.7%(36.9%)	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
grizzly	965,831	36.0%(35.3%)	Mcr	Jav	Svt	Non	Lin	Non	Lin	Raw	Rea	0
cutelyst-thread-epol	938,400	35.0%(34.3%)	Ful	C++	Qt	Non	Lin	Non	Lin	Raw	Rea	17
apistar	931,909	34.7%(34.0%)	Mcr	Py	Non	Mei	Lin	Non	Lin	Raw	Rea	0
revenj-jvm	884,318	32.9%(32.3%)	Ful	Jav	Svt	Res	Lin	Pg	Lin	Ful	Rea	0
gemini	824,347	30.7%(30.1%)	Ful	Jav	Svt	Res	Lin	Non	Lin	Mcr	Rea	0
wheezyweb	824,132	30.7%(30.1%)	Mcr	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
go	752,853	28.0%(27.5%)	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
gin	750,780	28.0%(27.4%)	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
servlet	730,095	27.2%(26.7%)	Plt	Jav	Svt	Res	Lin	Non	Lin	Raw	Rea	0
finatra	710,811	26.5%(26.0%)	Mcr	Sca	Nty	Non	Lin	Non	Lin	Raw	Rea	3,358
falcon	695,897	25.9%(25.4%)	Mcr	Py	Non	Mei	Lin	Non	Lin	Raw	Rea	0
kami	686,682	25.6%(25.1%)	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
uvicorn	624,188	23.3%(22.8%)	Plt	Py	Non	Non	Lin	Pg	Lin	Raw	Rea	0
falcon-py3	610,525	22.7%(22.3%)	Mcr	Py	Non	Mei	Lin	Non	Lin	Raw	Rea	0
beego	604,659	22.5%(22.1%)	Ful	Go	Non	Non	Lin	Non	Lin	Raw	Rea	0
finagle	556,888	20.7%(20.3%)	Mcr	Sca	Nty	Non	Lin	Non	Lin	Raw	Rea	754
fintrospect	548,336	20.4%(20.0%)	Mcr	Sca	Nty	Non	Lin	My	Lin	Raw	Rea	782
restexpress	497,486	18.5%(18.2%)	Mcr	Jav	Nty	Non	Lin	Mo	Lin	Raw	Rea	7,071
blade	490,441	18.3%(17.9%)	Ful	Jav	Non	Non	Lin	My	Lin	Ful	Rea	0
akka-http	490,038	18.3%(17.9%)	Mcr	Sca	Akk	Non	Lin	My	Lin	Raw	Rea	2,254
finch	477,543	17.8%(17.4%)	Mcr	Sca	Nty	Non	Lin	Non	Lin	Raw	Rea	3,742
bottle	462,141	17.2%(16.9%)	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
bottle-py3	406,822	15.2%(14.9%)	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
jlhttp	386,636	14.4%(14.1%)	Plt	Jav	JLH	Non	Lin	Non	Lin	Raw	Rea	646
treefrog-hybrid	382,951	14.3%(14.0%)	Ful	C++	Non	Non	Lin	My	Lin	Mcr	Rea	539
weppy	365,421	13.6%(13.3%)	Ful	Py	Mei	Non	Lin	Pg	Lin	Ful	Rea	0
weppy-py3	363,672	13.5%(13.3%)	Ful	Py	Mei	Non	Lin	Pg	Lin	Ful	Rea	0
pyramid-py2	331,732	12.4%(12.1%)	Ful	Py	Non	Mei	Lin	Pg	Lin	Ful	Rea	0
falcore	331,164	12.3%(12.1%)	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
pyramid	310,715	11.6%(11.3%)	Ful	Py	Non	Mei	Lin	Pg	Lin	Ful	Rea	0
jetty	305,109	11.4%(11.1%)	Plt	Jav	Jty	Non	Lin	Non	Lin	Raw	Rea	584
jooby	294,958	11.0%(10.8%)	Ful	Jav	Nty	Non	Lin	Pg	Lin	Raw	Rea	0
spray	294,030	11.0%(10.7%)	Mcr	Sca	Akk	Non	Lin	Non	Lin	Raw	Rea	0
wicket	271,565	10.1%(9.9%)	Ful	Jav	Svt	Res	Lin	My	Lin	Ful	Rea	0
jetty-servlet	262,519	9.8%(9.6%)	Plt	Jav	Jty	Non	Lin	Non	Lin	Raw	Rea	4,788
morepath	256,610	9.6%(9.4%)	Mcr	Py	Mei	Gun	Lin	Pg	Lin	Ful	Rea	0
minijax	226,826	8.4%(8.3%)	Ful	Jav	JAX	Jty	Lin	My	Lin	Ful	Rea	4,270
play2-java-netty	223,814	8.3%(8.2%)	Ful	Jav	Nty	Non	Lin	Non	Lin	Raw	Rea	0
falcon-pypy2	221,939	8.3%(8.1%)	Mcr	Py	Non	Tor	Lin	Non	Lin	Raw	Rea	0
play2-scala	221,514	8.3%(8.1%)	Ful	Sca	Akk	Non	Lin	Non	Lin	Raw	Rea	0
treefrog-postgres	219,856	8.2%(8.0%)	Ful	C++	Non	Non	Lin	Pg	Lin	Mcr	Rea	2,519
jawn	219,577	8.2%(8.0%)	Ful	Jav	Svt	Utw	Lin	Pg	Lin	Raw	Rea	0
treefrog-mongodb	217,082	8.1%(7.9%)	Ful	C++	Non	Non	Lin	Mo	Lin	Mcr	Rea	40
treefrog	213,423	8.0%(7.8%)	Ful	C++	Non	Non	Lin	My	Lin	Mcr	Rea	43
tornado-pypy2	201,831	7.5%(7.4%)	Plt	Py	Non	Tor	Lin	Mo	Lin	Raw	Rea	0
aiohttp	191,436	7.1%(7.0%)	Mcr	Py	asy	Gun	Lin	Pg	Lin	Ful	Rea	0
grizzly-jersey	190,532	7.1%(7.0%)	Mcr	Jav	JAX	Non	Lin	My	Lin	Ful	Rea	0
silicon	172,714	6.4%(6.3%)	Mcr	C++	Non	mic	Lin	My	Lin	Ful	Rea	53,781
undertow-jersey-hika	170,106	6.3%(6.2%)	Plt	Jav	JAX	Non	Lin	My	Lin	Ful	Rea	863
django-py3	159,900	6.0%(5.8%)	Ful	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
undertow-jersey-c3p0	154,758	5.8%(5.7%)	Plt	Jav	JAX	Non	Lin	My	Lin	Ful	Rea	0
play2-scala-scala-ne	154,544	5.8%(5.6%)	Ful	Sca	Nty	Non	Lin	Non	Lin	Raw	Rea	0
echo	150,523	5.6%(5.5%)	Mcr	Go	Non	Non	Lin	Pg	Lin	Raw	Rea	0
tapestry	136,018	5.1%(5.0%)	Ful	Jav	Svt	Res	Lin	My	Lin	Ful	Rea	0
spring	131,249	4.9%(4.8%)	Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	0
activeweb	129,324	4.8%(4.7%)	Ful	Jav	Svt	Non	Lin	My	Lin	Mcr	Rea	0
api_hour	123,385	4.6%(4.5%)	Mcr	Py	asy	Gun	Lin	Pg	Lin	Raw	Rea	0
flask-py3	121,901	4.5%(4.5%)	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
comsat-servlet-under	121,743	4.5%(4.4%)	Plt	Jav	Svt	Non	Lin	Non	Lin	Raw	Rea	0
curacao	119,759	4.5%(4.4%)	Mcr	Jav	Svt	Non	Lin	Non	Lin	Raw	Rea	3
revel	117,584	4.4%(4.3%)	Ful	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
flask	113,578	4.2%(4.1%)	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
wt	111,114	4.1%(4.1%)	Ful	C++	Non	Non	Lin	My	Lin	Ful	Rea	0
cutelyst	106,310	4.0%(3.9%)	Ful	C++	Qt	Non	Lin	Non	Lin	Raw	Rea	0
poco	102,247	3.8%(3.7%)	Plt	C++	Non	Non	Lin	Non	Lin	Raw	Rea	233,647
comsat-servlet	99,576	3.7%(3.6%)	Plt	Jav	Svt	Non	Lin	Non	Lin	Raw	Rea	0
bottle-pypy2	95,722	3.6%(3.5%)	Mcr	Py	Tor	Non	Lin	My	Lin	Ful	Rea	0
dropwizard	94,713	3.5%(3.5%)	Ful	Jav	JAX	Jty	Lin	Non	Lin	Ful	Rea	0
turbogears	88,620	3.3%(3.2%)	Mcr	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
http4s	87,193	3.2%(3.2%)	Mcr	Sca	NIO	bla	Lin	Pg	Lin	Raw	Rea	931
cutelyst-nginx	85,448	3.2%(3.1%)	Ful	C++	Qt	ngx	Lin	Non	Lin	Raw	Rea	0
comsat-webactors-und	81,287	3.0%(3.0%)	Plt	Jav	Com	Non	Lin	Non	Lin	Raw	Rea	0
django	77,134	2.9%(2.8%)	Ful	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
play2	70,167	2.6%(2.6%)	Ful	Jav	Akk	Non	Lin	Non	Lin	Raw	Rea	1,911
wildfly-ee7	66,156	2.5%(2.4%)	Ful	Jav	Svt	Wil	Lin	My	Lin	Ful	Rea	3
tornado	61,738	2.3%(2.3%)	Plt	Py	Non	Tor	Lin	Mo	Lin	Raw	Rea	0
cutelyst-pf-epoll	61,383	2.3%(2.2%)	Ful	C++	Qt	Non	Lin	Non	Lin	Raw	Rea	0
tornado-py3	55,282	2.1%(2.0%)	Plt	Py	Non	Tor	Lin	Mo	Lin	Raw	Rea	0
comsat-webactors	51,727	1.9%(1.9%)	Plt	Jav	Com	Non	Lin	Non	Lin	Raw	Rea	0
web2py-optimized	50,373	1.9%(1.8%)	Ful	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
aah	44,783	1.7%(1.6%)	Ful	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
scruffy	41,366	1.5%(1.5%)	Mcr	Sca	Utw	Non	Lin	Mo	Lin	Raw	Rea	5,629
weppy-pypy2	22,764	0.8%(0.8%)	Ful	Py	Tor	Non	Lin	Pg	Lin	Ful	Rea	0
goji	22,435	0.8%(0.8%)	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
web2py	20,143	0.8%(0.7%)	Ful	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
cppcms	15,665	0.6%(0.6%)	Plt	C++	Non	Non	Lin	My	Lin	Raw	Rea	0
cppcms-postgres	15,597	0.6%(0.6%)	Plt	C++	Non	Non	Lin	Pg	Lin	Raw	Rea	0
tornado-postgresql-r	13,267	0.5%(0.5%)	Plt	Py	Non	Tor	Lin	Pg	Lin	Raw	Rea	0
ninja-standalone	13,069	0.5%(0.5%)	Ful	Jav	Jty	Non	Lin	My	Lin	Ful	Rea	568
flask-pypy2	11,316	0.4%(0.4%)	Mcr	Py	Non	Tor	Lin	My	Lin	Ful	Rea	0
cppcms-direct	10,587	0.4%(0.4%)	Plt	C++	Non	Non	Lin	My	Lin	Raw	Rea	3
flask-nginx-uwsgi	4,868	0.2%(0.2%)	Mcr	Py	Non	ngx	Lin	My	Lin	Ful	Rea	0
weppy-nginx-uwsgi	4,752	0.2%(0.2%)	Ful	Py	uWS	ngx	Lin	Pg	Lin	Ful	Rea	0
bottle-nginx-uwsgi	4,735	0.2%(0.2%)	Mcr	Py	uWS	ngx	Lin	My	Lin	Ful	Rea	0
spark	2,604	0.1%(0.1%)	Mcr	Jav	Svt	Res	Lin	My	Lin	Ful	Rea	0
klein	781	0.0%(0.0%)	Mcr	Py	Non	Twi	Lin	My	Lin	Ful	Rea	26
webware	730	0.0%(0.0%)	Mcr	Py	Non	Non	Lin	My	Lin	Ful	Rea	266
cherrypy-py3	493	0.0%(0.0%)	Mcr	Py	Non	Non	Lin	My	Lin	Ful	Rea	699
cherrypy	298	0.0%(0.0%)	Mcr	Py	Non	Non	Lin	My	Lin	Ful	Rea	598
sanic	21	0.0%(0.0%)	Mcr	Py	Non	Non	Lin	Non	Lin	Raw	Rea	656
ffead-cpp	0	0.0%(0.0%)	Ful	C++	ffe	ffe	Lin	Mo	Lin	Ful	Rea	0
ffead-cpp-apache-mon	0	0.0%(0.0%)	Ful	C++	ffe	Apc	Lin	Mo	Lin	Ful	Rea	0
ffead-cpp-apache-mys	0	0.0%(0.0%)	Ful	C++	ffe	Apc	Lin	My	Lin	Ful	Rea	0
ffead-cpp-apache-pos	0	0.0%(0.0%)	Ful	C++	ffe	Apc	Lin	Pg	Lin	Ful	Rea	0
ffead-cpp-mysql	0	0.0%(0.0%)	Ful	C++	ffe	ffe	Lin	My	Lin	Ful	Rea	0
ffead-cpp-nginx-mong	0	0.0%(0.0%)	Ful	C++	ffe	ngx	Lin	Mo	Lin	Ful	Rea	0
ffead-cpp-nginx-mysq	0	0.0%(0.0%)	Ful	C++	ffe	ngx	Lin	My	Lin	Ful	Rea	0
ffead-cpp-nginx-post	0	0.0%(0.0%)	Ful	C++	ffe	ngx	Lin	Pg	Lin	Ful	Rea	0
ffead-cpp-postgresql	0	0.0%(0.0%)	Ful	C++	ffe	ffe	Lin	Pg	Lin	Ful	Rea	0
luna	—	Did not complete	Ful	C++	Non	Non	Lin	Non	Lin	Raw	Rea	—
luna-epoll	—	Did not complete	Ful	C++	Non	Non	Lin	Non	Lin	Raw	Rea	—
luna-thread-per-conn	—	Did not complete	Ful	C++	Non	Non	Lin	Non	Lin	Raw	Rea	—
wheezyweb-py3	—	Did not complete	Mcr	Py	Non	Mei	Lin	My	Lin	Ful	Rea	—

Plaintext responses per second, Dell servers at ServerCentral

Framework	256	1,024	4,096	16,384	Best	Cls	Lng	Plt	FE	Aos	DB	Dos	Orm	IA	Errors
aah	44,783	44,196	44,491	44,647	44,783	Ful	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
actframework	1,883,784	2,424,671	2,172,084	1,377,626	2,424,671	Ful	Jav	Utw	Non	Lin	Non	Lin	Ful	Rea	0
activeweb	126,963	129,324	110,178	109,193	129,324	Ful	Jav	Svt	Non	Lin	My	Lin	Mcr	Rea	0
aiohttp	191,436	183,896	167,232	160,864	191,436	Mcr	Py	asy	Gun	Lin	Pg	Lin	Ful	Rea	0
akka-http	381,176	365,210	483,034	490,038	490,038	Mcr	Sca	Akk	Non	Lin	My	Lin	Raw	Rea	2,254
apistar	778,198	931,909	865,096	856,448	931,909	Mcr	Py	Non	Mei	Lin	Non	Lin	Raw	Rea	0
api_hour	123,385	119,163	112,495	112,229	123,385	Mcr	Py	asy	Gun	Lin	Pg	Lin	Raw	Rea	0
blade	471,308	490,441	485,280	463,334	490,441	Ful	Jav	Non	Non	Lin	My	Lin	Ful	Rea	0
blaze	993,687	831,825	1,324,377	1,362,644	1,362,644	Mcr	Sca	bla	Non	Lin	Non	Lin	Raw	Rea	2,638
bottle	454,386	462,141	459,524	452,469	462,141	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
bottle-nginx-uwsgi	4,735	4,587	4,181	3,312	4,735	Mcr	Py	uWS	ngx	Lin	My	Lin	Ful	Rea	0
bottle-py3	395,648	406,822	390,775	369,430	406,822	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
bottle-pypy2	63,701	81,251	95,722	94,063	95,722	Mcr	Py	Tor	Non	Lin	My	Lin	Ful	Rea	0
colossus	1,356,745	1,440,236	1,405,369	1,319,786	1,440,236	Mcr	Sca	Akk	Non	Lin	Non	Lin	Raw	Rea	0
cpoll_cppsp	1,694,316	2,119,565	2,112,000	2,033,028	2,119,565	Plt	C++	Non	Non	Lin	Non	Lin	Raw	Rea	0
curacao	108,118	119,759	107,779	18,583	119,759	Mcr	Jav	Svt	Non	Lin	Non	Lin	Raw	Rea	3
cutelyst	75,984	105,287	105,202	106,310	106,310	Ful	C++	Qt	Non	Lin	Non	Lin	Raw	Rea	0
cutelyst-nginx	85,448	82,471	80,776	79,984	85,448	Ful	C++	Qt	ngx	Lin	Non	Lin	Raw	Rea	0
cutelyst-pf-epoll	56,617	61,383	55,804	41,937	61,383	Ful	C++	Qt	Non	Lin	Non	Lin	Raw	Rea	0
cutelyst-thread	88,581	353,679	1,032,448	567,765	1,032,448	Ful	C++	Qt	Non	Lin	Non	Lin	Raw	Rea	0
cutelyst-thread-epol	83,877	351,906	1,564,883	1,625,125	1,625,125	Ful	C++	Qt	Non	Lin	Non	Lin	Raw	Rea	0
cutelyst-thread-epol	81,532	347,672	912,158	938,400	938,400	Ful	C++	Qt	Non	Lin	Non	Lin	Raw	Rea	17
django	77,134	73,992	73,047	71,961	77,134	Ful	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
django-py3	159,900	159,665	159,111	155,396	159,900	Ful	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
dropwizard	75,717	76,868	88,269	94,713	94,713	Ful	Jav	JAX	Jty	Lin	Non	Lin	Ful	Rea	0
falcon	638,629	695,897	690,542	680,588	695,897	Mcr	Py	Non	Mei	Lin	Non	Lin	Raw	Rea	0
falcon-py3	601,449	610,525	609,501	596,159	610,525	Mcr	Py	Non	Mei	Lin	Non	Lin	Raw	Rea	0
falcon-pypy2	210,755	221,939	218,011	197,093	221,939	Mcr	Py	Non	Tor	Lin	Non	Lin	Raw	Rea	0
falcore	280,107	308,018	331,164	330,254	331,164	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
ffead-cpp	0	0	0	0	0	Ful	C++	ffe	ffe	Lin	Mo	Lin	Ful	Rea	0
ffead-cpp-apache-mon	0	0	0	0	0	Ful	C++	ffe	Apc	Lin	Mo	Lin	Ful	Rea	0
ffead-cpp-apache-mys	0	0	0	0	0	Ful	C++	ffe	Apc	Lin	My	Lin	Ful	Rea	0
ffead-cpp-apache-pos	0	0	0	0	0	Ful	C++	ffe	Apc	Lin	Pg	Lin	Ful	Rea	0
ffead-cpp-mysql	0	0	0	0	0	Ful	C++	ffe	ffe	Lin	My	Lin	Ful	Rea	0
ffead-cpp-nginx-mong	0	0	0	0	0	Ful	C++	ffe	ngx	Lin	Mo	Lin	Ful	Rea	0
ffead-cpp-nginx-mysq	0	0	0	0	0	Ful	C++	ffe	ngx	Lin	My	Lin	Ful	Rea	0
ffead-cpp-nginx-post	0	0	0	0	0	Ful	C++	ffe	ngx	Lin	Pg	Lin	Ful	Rea	0
ffead-cpp-postgresql	0	0	0	0	0	Ful	C++	ffe	ffe	Lin	Pg	Lin	Ful	Rea	0
finagle	464,303	556,888	549,171	527,786	556,888	Mcr	Sca	Nty	Non	Lin	Non	Lin	Raw	Rea	754
finatra	654,644	639,304	710,811	686,992	710,811	Mcr	Sca	Nty	Non	Lin	Non	Lin	Raw	Rea	3,358
finch	426,744	469,315	477,543	477,075	477,543	Mcr	Sca	Nty	Non	Lin	Non	Lin	Raw	Rea	3,742
fintrospect	517,139	548,336	547,647	538,744	548,336	Mcr	Sca	Nty	Non	Lin	My	Lin	Raw	Rea	782
flask	113,578	112,443	113,252	39,929	113,578	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
flask-nginx-uwsgi	4,868	4,821	4,426	3,549	4,868	Mcr	Py	Non	ngx	Lin	My	Lin	Ful	Rea	0
flask-py3	121,901	120,890	119,631	47,396	121,901	Mcr	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
flask-pypy2	10,100	9,976	11,316	7,260	11,316	Mcr	Py	Non	Tor	Lin	My	Lin	Ful	Rea	0
gemini	678,004	824,347	822,893	808,414	824,347	Ful	Jav	Svt	Res	Lin	Non	Lin	Mcr	Rea	0
gin	605,826	721,486	750,780	647,205	750,780	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
goji	22,163	21,067	22,435	21,578	22,435	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
grizzly	840,803	937,268	965,831	956,690	965,831	Mcr	Jav	Svt	Non	Lin	Non	Lin	Raw	Rea	0
grizzly-jersey	163,269	180,045	190,532	180,690	190,532	Mcr	Jav	JAX	Non	Lin	My	Lin	Ful	Rea	0
http4s	86,829	87,193	85,430	85,369	87,193	Mcr	Sca	NIO	bla	Lin	Pg	Lin	Raw	Rea	931
japronto	2,305,035	2,356,964	2,280,180	2,216,809	2,356,964	Mcr	Py	Non	Non	Lin	Non	Lin	Raw	Rea	178
jawn	219,577	218,211	197,206	157,852	219,577	Ful	Jav	Svt	Utw	Lin	Pg	Lin	Raw	Rea	0
undertow-jersey-c3p0	154,758	149,289	100,358	60,094	154,758	Plt	Jav	JAX	Non	Lin	My	Lin	Ful	Rea	0
undertow-jersey-hika	167,144	155,908	170,106	85,950	170,106	Plt	Jav	JAX	Non	Lin	My	Lin	Ful	Rea	863
jooby	294,958	282,234	274,329	224,609	294,958	Ful	Jav	Nty	Non	Lin	Pg	Lin	Raw	Rea	0
kami	643,562	679,200	686,682	560,723	686,682	Mcr	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
klein	781	715	585	682	781	Mcr	Py	Non	Twi	Lin	My	Lin	Ful	Rea	26
luna	— Did not complete —					Ful	C++	Non	Non	Lin	Non	Lin	Raw	Rea	—
luna-epoll	— Did not complete —					Ful	C++	Non	Non	Lin	Non	Lin	Raw	Rea	—
luna-thread-per-conn	— Did not complete —					Ful	C++	Non	Non	Lin	Non	Lin	Raw	Rea	—
minijax	211,182	215,942	214,704	226,826	226,826	Ful	Jav	JAX	Jty	Lin	My	Lin	Ful	Rea	4,270
morepath	256,610	253,937	251,037	120,685	256,610	Mcr	Py	Mei	Gun	Lin	Pg	Lin	Ful	Rea	0
netty	1,398,537	2,426,223	2,440,869	2,447,315	2,447,315	Plt	Jav	Nty	Non	Lin	Non	Lin	Raw	Rea	1
ninja-standalone	13,036	13,069	12,670	12,786	13,069	Ful	Jav	Jty	Non	Lin	My	Lin	Ful	Rea	568
bayou	1,608,378	1,732,066	1,752,492	1,670,074	1,752,492	Plt	Jav	Bay	Non	Lin	Non	Lin	Raw	Rea	4,674
beego	470,819	559,166	604,659	487,611	604,659	Ful	Go	Non	Non	Lin	Non	Lin	Raw	Rea	0
cherrypy	298	296	271	246	298	Mcr	Py	Non	Non	Lin	My	Lin	Ful	Rea	598
cherrypy-py3	493	481	485	444	493	Mcr	Py	Non	Non	Lin	My	Lin	Ful	Rea	699
comsat-servlet	99,576	40,060	58,758	93,788	99,576	Plt	Jav	Svt	Non	Lin	Non	Lin	Raw	Rea	0
comsat-servlet-under	107,193	109,269	121,743	30,488	121,743	Plt	Jav	Svt	Non	Lin	Non	Lin	Raw	Rea	0
comsat-webactors	51,545	31,673	40,937	51,727	51,727	Plt	Jav	Com	Non	Lin	Non	Lin	Raw	Rea	0
comsat-webactors-und	81,287	81,176	14,759	60,665	81,287	Plt	Jav	Com	Non	Lin	Non	Lin	Raw	Rea	0
cppcms	15,665	14,907	15,418	15,005	15,665	Plt	C++	Non	Non	Lin	My	Lin	Raw	Rea	0
cppcms-direct	10,570	10,587	10,219	9,842	10,587	Plt	C++	Non	Non	Lin	My	Lin	Raw	Rea	3
cppcms-postgres	15,597	14,596	15,279	14,746	15,597	Plt	C++	Non	Non	Lin	Pg	Lin	Raw	Rea	0
echo	148,912	150,523	147,825	140,124	150,523	Mcr	Go	Non	Non	Lin	Pg	Lin	Raw	Rea	0
fasthttp	2,665,278	2,614,265	2,582,683	2,560,206	2,665,278	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
go	527,944	705,557	752,853	604,064	752,853	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
go-prefork	698,192	993,370	999,166	1,011,589	1,011,589	Plt	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
jetty	280,633	305,109	300,796	296,413	305,109	Plt	Jav	Jty	Non	Lin	Non	Lin	Raw	Rea	584
jetty-servlet	228,320	234,322	234,458	262,519	262,519	Plt	Jav	Jty	Non	Lin	Non	Lin	Raw	Rea	4,788
jlhttp	364,312	386,636	373,849	325,803	386,636	Plt	Jav	JLH	Non	Lin	Non	Lin	Raw	Rea	646
light-4j	1,930,192	1,845,583	2,418,083	2,435,150	2,435,150	Plt	Jav	Lig	Non	Lin	Non	Lin	Raw	Rea	0
poco	102,247	101,307	100,398	98,599	102,247	Plt	C++	Non	Non	Lin	Non	Lin	Raw	Rea	233,647
revel	96,623	101,053	105,350	117,584	117,584	Ful	Go	Non	Non	Lin	My	Lin	Raw	Rea	0
servlet	569,319	730,095	719,057	709,436	730,095	Plt	Jav	Svt	Res	Lin	Non	Lin	Raw	Rea	0
tornado	61,549	61,738	57,772	58,474	61,738	Plt	Py	Non	Tor	Lin	Mo	Lin	Raw	Rea	0
tornado-postgresql-r	13,267	13,083	12,794	11,503	13,267	Plt	Py	Non	Tor	Lin	Pg	Lin	Raw	Rea	0
tornado-py3	55,282	54,797	54,196	52,194	55,282	Plt	Py	Non	Tor	Lin	Mo	Lin	Raw	Rea	0
tornado-pypy2	201,831	191,736	180,891	169,326	201,831	Plt	Py	Non	Tor	Lin	Mo	Lin	Raw	Rea	0
undertow	1,953,575	2,544,955	2,525,038	2,356,475	2,544,955	Plt	Jav	Utw	Non	Lin	Non	Lin	Raw	Rea	0
vertx	2,105,562	2,381,369	2,447,022	2,503,741	2,503,741	Plt	Jav	Ver	Non	Lin	Non	Lin	Raw	Rea	0
wildfly-ee7	66,156	57,866	48,257	31,242	66,156	Ful	Jav	Svt	Wil	Lin	My	Lin	Ful	Rea	3
play2	67,774	68,474	70,167	69,839	70,167	Ful	Jav	Akk	Non	Lin	Non	Lin	Raw	Rea	1,911
play2-java-netty	223,814	137,476	132,849	131,022	223,814	Ful	Jav	Nty	Non	Lin	Non	Lin	Raw	Rea	0
play2-scala	221,514	209,484	211,909	200,186	221,514	Ful	Sca	Akk	Non	Lin	Non	Lin	Raw	Rea	0
play2-scala-scala-ne	154,544	19,147	21,845	21,036	154,544	Ful	Sca	Nty	Non	Lin	Non	Lin	Raw	Rea	0
proteus	1,733,177	1,306,166	2,376,080	2,441,688	2,441,688	Mcr	Jav	Utw	Non	Lin	Pg	Lin	Raw	Rea	0
pyramid	310,715	308,801	305,968	297,578	310,715	Ful	Py	Non	Mei	Lin	Pg	Lin	Ful	Rea	0
pyramid-py2	330,625	331,732	328,244	320,306	331,732	Ful	Py	Non	Mei	Lin	Pg	Lin	Ful	Rea	0
rapidoid	2,567,818	2,608,986	2,557,081	2,534,979	2,608,986	Plt	Jav	Rap	Non	Lin	Non	Lin	Raw	Rea	0
rapidoid-http-fast	2,582,458	2,654,663	2,594,930	2,571,448	2,654,663	Plt	Jav	Rap	Non	Lin	Non	Lin	Raw	Rea	0
restexpress	412,690	491,746	497,486	462,662	497,486	Mcr	Jav	Nty	Non	Lin	Mo	Lin	Raw	Rea	7,071
revenj-jvm	740,764	884,318	867,007	814,061	884,318	Ful	Jav	Svt	Res	Lin	Pg	Lin	Ful	Rea	0
s-server	720,875	1,504,833	1,493,233	1,560,315	1,560,315	Plt	Sca	s-s	Non	Lin	Non	Lin	Raw	Rea	0
sanic	21	14	3	0	21	Mcr	Py	Non	Non	Lin	Non	Lin	Raw	Rea	656
scruffy	337	8,837	26,176	41,366	41,366	Mcr	Sca	Utw	Non	Lin	Mo	Lin	Raw	Rea	5,629
silicon	94,621	163,551	169,358	172,714	172,714	Mcr	C++	Non	mic	Lin	My	Lin	Ful	Rea	53,781
spark	2,457	1,626	1,636	2,604	2,604	Mcr	Jav	Svt	Res	Lin	My	Lin	Ful	Rea	0
spray	294,030	277,515	290,512	283,708	294,030	Mcr	Sca	Akk	Non	Lin	Non	Lin	Raw	Rea	0
spring	131,249	130,717	105,894	49,498	131,249	Ful	Jav	Utw	Non	Lin	My	Lin	Ful	Rea	0
tapestry	136,018	129,375	126,365	131,974	136,018	Ful	Jav	Svt	Res	Lin	My	Lin	Ful	Rea	0
treefrog	213,423	208,385	210,597	212,675	213,423	Ful	C++	Non	Non	Lin	My	Lin	Mcr	Rea	43
treefrog-hybrid	346,224	382,951	370,906	360,534	382,951	Ful	C++	Non	Non	Lin	My	Lin	Mcr	Rea	539
treefrog-mongodb	217,082	209,614	209,445	211,373	217,082	Ful	C++	Non	Non	Lin	Mo	Lin	Mcr	Rea	40
treefrog-postgres	209,674	208,639	211,269	219,856	219,856	Ful	C++	Non	Non	Lin	Pg	Lin	Mcr	Rea	2,519
turbogears	88,185	88,388	88,620	43,723	88,620	Mcr	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
ulib	2,679,274	2,666,076	2,594,403	2,571,793	2,679,274	Plt	C++	Non	ULi	Lin	Non	Lin	Mcr	Rea	2
ulib-plaintext_fit	2,684,333	2,643,658	2,598,313	2,573,619	2,684,333	Plt	C++	Non	ULi	Lin	Non	Lin	Mcr	Rea	2
uvicorn	591,105	624,188	615,461	580,966	624,188	Plt	Py	Non	Non	Lin	Pg	Lin	Raw	Rea	0
vertx-web	1,858,548	2,411,118	2,315,799	2,221,333	2,411,118	Mcr	Jav	vtx	Non	Lin	Non	Lin	Raw	Rea	0
web2py	20,143	19,707	17,481	6,085	20,143	Ful	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
web2py-optimized	50,373	49,694	43,038	10,953	50,373	Ful	Py	Mei	Non	Lin	My	Lin	Ful	Rea	0
webware	274	728	730	164	730	Mcr	Py	Non	Non	Lin	My	Lin	Ful	Rea	266
weppy	365,421	328,929	330,448	319,389	365,421	Ful	Py	Mei	Non	Lin	Pg	Lin	Ful	Rea	0
weppy-nginx-uwsgi	4,752	4,571	4,183	3,392	4,752	Ful	Py	uWS	ngx	Lin	Pg	Lin	Ful	Rea	0
weppy-py3	322,562	327,271	363,672	319,334	363,672	Ful	Py	Mei	Non	Lin	Pg	Lin	Ful	Rea	0
weppy-pypy2	14,161	20,064	22,764	19,608	22,764	Ful	Py	Tor	Non	Lin	Pg	Lin	Ful	Rea	0
wheezyweb	652,390	814,985	824,132	806,922	824,132	Mcr	Py	Non	Mei	Lin	My	Lin	Ful	Rea	0
wheezyweb-py3	— Did not complete —					Mcr	Py	Non	Mei	Lin	My	Lin	Ful	Rea	—
wicket	185,102	271,565	264,242	260,798	271,565	Ful	Jav	Svt	Res	Lin	My	Lin	Ful	Rea	0
wt	111,114	109,294	108,935	108,385	111,114	Ful	C++	Non	Non	Lin	My	Lin	Ful	Rea	0

Latency of plaintext responses, Dell servers at ServerCentral

Framework	Average latency (lower is better)		σ (SD)	Max	Errors
poco	12.7 ms	0.3%	6.4 ms	236.1 ms	233,647
bayou	28.1 ms	0.7%	42.2 ms	1460.0 ms	4,674
silicon	46.6 ms	1.2%	14.8 ms	641.5 ms	53,781
japronto	53.6 ms	1.4%	146.3 ms	2300.0 ms	178
vertx-web	61.2 ms	1.5%	67.5 ms	4090.0 ms	0
finagle	87.1 ms	2.2%	108.9 ms	3570.0 ms	754
colossus	88.0 ms	2.2%	74.8 ms	1980.0 ms	0
uvicorn	88.8 ms	2.2%	71.8 ms	1550.0 ms	0
treefrog-hybrid	100.1 ms	2.5%	156.2 ms	3530.0 ms	539
cutelyst	108.7 ms	2.8%	366.1 ms	7250.0 ms	0
go-prefork	109.1 ms	2.8%	94.4 ms	2720.0 ms	0
fintrospect	127.7 ms	3.2%	117.0 ms	1670.0 ms	782
finatra	129.8 ms	3.3%	186.1 ms	6490.0 ms	3,358
finch	145.9 ms	3.7%	179.1 ms	3820.0 ms	3,742
play2	158.7 ms	4.0%	643.2 ms	7880.0 ms	1,911
ulib	159.0 ms	4.0%	218.1 ms	7710.0 ms	2
cutelyst-thread	167.6 ms	4.2%	571.3 ms	7960.0 ms	0
cutelyst-thread-epol	173.1 ms	4.4%	593.9 ms	7990.0 ms	0
blaze	175.0 ms	4.4%	409.8 ms	7430.0 ms	2,638
jetty-servlet	186.4 ms	4.7%	114.5 ms	2370.0 ms	4,788
cutelyst-pf-epoll	192.0 ms	4.9%	604.2 ms	7120.0 ms	0
apistar	192.0 ms	4.9%	301.7 ms	7020.0 ms	0
gemini	194.3 ms	4.9%	539.5 ms	7780.0 ms	0
spray	196.1 ms	5.0%	496.3 ms	7760.0 ms	0
play2-scala	196.7 ms	5.0%	768.2 ms	8000.0 ms	0
light-4j	197.4 ms	5.0%	338.7 ms	7890.0 ms	0
cpoll_cppsp	204.2 ms	5.2%	382.5 ms	7980.0 ms	0
s-server	206.8 ms	5.2%	408.8 ms	7850.0 ms	0
vertx	210.3 ms	5.3%	304.5 ms	7760.0 ms	0
ulib-plaintext_fit	210.7 ms	5.3%	349.2 ms	7990.0 ms	2
treefrog-mongodb	212.9 ms	5.4%	166.1 ms	2340.0 ms	40
proteus	215.2 ms	5.4%	365.7 ms	7900.0 ms	0
rapidoid	217.3 ms	5.5%	291.3 ms	7380.0 ms	0
wheezyweb	218.1 ms	5.5%	229.6 ms	4790.0 ms	0
actframework	236.3 ms	6.0%	278.3 ms	5360.0 ms	0
treefrog	241.4 ms	6.1%	187.0 ms	2590.0 ms	43
grizzly	244.0 ms	6.2%	914.6 ms	7990.0 ms	0
treefrog-postgres	244.3 ms	6.2%	190.6 ms	2890.0 ms	2,519
rapidoid-http-fast	244.5 ms	6.2%	339.9 ms	7840.0 ms	0
netty	247.0 ms	6.3%	326.7 ms	5240.0 ms	1
revenj-jvm	252.7 ms	6.4%	677.4 ms	7780.0 ms	0
jlhttp	253.6 ms	6.4%	215.8 ms	3580.0 ms	646
akka-http	256.1 ms	6.5%	794.9 ms	7310.0 ms	2,254
falcon	257.2 ms	6.5%	394.9 ms	7950.0 ms	0
cutelyst-thread-epol	261.5 ms	6.6%	804.9 ms	7920.0 ms	17
restexpress	270.7 ms	6.9%	378.9 ms	6670.0 ms	7,071
falcon-py3	272.6 ms	6.9%	339.4 ms	7770.0 ms	0
servlet	284.1 ms	7.2%	840.5 ms	7980.0 ms	0
jetty	290.8 ms	7.4%	306.0 ms	3720.0 ms	584
fasthttp	295.0 ms	7.5%	454.4 ms	7960.0 ms	0
undertow	322.8 ms	8.2%	477.9 ms	7850.0 ms	0
wt	325.9 ms	8.2%	539.4 ms	7980.0 ms	0
wicket	328.2 ms	8.3%	699.7 ms	7920.0 ms	0
bottle	339.3 ms	8.6%	466.6 ms	7880.0 ms	0
cherrypy-py3	360.6 ms	9.1%	154.9 ms	1340.0 ms	699
gin	383.7 ms	9.7%	542.0 ms	7550.0 ms	0
tapestry	388.5 ms	9.8%	803.7 ms	7970.0 ms	0
play2-java-netty	396.0 ms	10.0%	872.0 ms	7990.0 ms	0
aiohttp	397.3 ms	10.1%	319.1 ms	2740.0 ms	0
minijax	425.9 ms	10.8%	449.5 ms	7140.0 ms	4,270
falcon-pypy2	450.6 ms	11.4%	244.8 ms	2110.0 ms	0
tornado-pypy2	459.5 ms	11.6%	240.5 ms	2010.0 ms	0
pyramid	462.0 ms	11.7%	492.9 ms	7640.0 ms	0
blade	478.5 ms	12.1%	699.8 ms	7950.0 ms	0
weppy	480.6 ms	12.2%	420.8 ms	7980.0 ms	0
api_hour	484.5 ms	12.3%	340.9 ms	2780.0 ms	0
bottle-py3	487.8 ms	12.4%	571.3 ms	7990.0 ms	0
jooby	488.0 ms	12.4%	288.9 ms	2230.0 ms	0
pyramid-py2	489.6 ms	12.4%	531.2 ms	7950.0 ms	0
weppy-py3	499.1 ms	12.6%	519.5 ms	7560.0 ms	0
http4s	549.0 ms	13.9%	241.7 ms	2480.0 ms	931
kami	557.4 ms	14.1%	876.8 ms	7390.0 ms	0
jawn	623.5 ms	15.8%	417.7 ms	7310.0 ms	0
cherrypy	649.4 ms	16.4%	462.9 ms	2840.0 ms	598
grizzly-jersey	665.5 ms	16.8%	392.4 ms	3170.0 ms	0
play2-scala-scala-ne	674.5 ms	17.1%	802.2 ms	6880.0 ms	0
spark	674.5 ms	17.1%	1500.0 ms	7730.0 ms	0
activeweb	745.7 ms	18.9%	1190.0 ms	7990.0 ms	0
falcore	801.5 ms	20.3%	717.6 ms	7940.0 ms	0
django-py3	857.7 ms	21.7%	720.2 ms	7980.0 ms	0
echo	920.3 ms	23.3%	477.1 ms	2520.0 ms	0
go	925.8 ms	23.4%	1340.0 ms	8000.0 ms	0
bottle-pypy2	971.4 ms	24.6%	671.0 ms	7460.0 ms	0
beego	1020.0 ms	25.8%	1450.0 ms	8000.0 ms	0
scruffy	1030.0 ms	26.1%	1350.0 ms	7260.0 ms	5,629
morepath	1060.0 ms	26.8%	1320.0 ms	7970.0 ms	0
comsat-servlet-under	1240.0 ms	31.4%	409.1 ms	3350.0 ms	0
curacao	1330.0 ms	33.7%	776.2 ms	7220.0 ms	3
revel	1330.0 ms	33.7%	948.3 ms	7780.0 ms	0
tornado	1370.0 ms	34.7%	664.0 ms	4110.0 ms	0
comsat-webactors-und	1390.0 ms	35.2%	674.8 ms	4210.0 ms	0
django	1590.0 ms	40.3%	1100.0 ms	7800.0 ms	0
comsat-webactors	1620.0 ms	41.0%	2130.0 ms	7940.0 ms	0
turbogears	1790.0 ms	45.3%	1460.0 ms	7990.0 ms	0
flask-py3	1810.0 ms	45.8%	1670.0 ms	8000.0 ms	0
cutelyst-nginx	1840.0 ms	46.6%	1030.0 ms	7100.0 ms	0
undertow-jersey-hika	1880.0 ms	47.6%	1550.0 ms	7680.0 ms	863
flask	1930.0 ms	48.9%	1700.0 ms	8000.0 ms	0
web2py-optimized	2010.0 ms	50.9%	1770.0 ms	8000.0 ms	0
comsat-servlet	2030.0 ms	51.4%	1490.0 ms	8000.0 ms	0
aah	2160.0 ms	54.7%	1610.0 ms	8000.0 ms	0
undertow-jersey-c3p0	2300.0 ms	58.2%	1610.0 ms	6920.0 ms	0
tornado-py3	2390.0 ms	60.5%	1300.0 ms	8000.0 ms	0
dropwizard	2450.0 ms	62.0%	1750.0 ms	8000.0 ms	0
spring	2690.0 ms	68.1%	1790.0 ms	8000.0 ms	0
ninja-standalone	2890.0 ms	73.2%	1390.0 ms	7810.0 ms	568
cppcms-postgres	3190.0 ms	80.8%	1960.0 ms	8000.0 ms	0
wildfly-ee7	3230.0 ms	81.8%	1780.0 ms	8000.0 ms	3
flask-pypy2	3330.0 ms	84.3%	2500.0 ms	7980.0 ms	0
weppy-pypy2	3470.0 ms	87.8%	1660.0 ms	7980.0 ms	0
cppcms	3590.0 ms	90.9%	1730.0 ms	7990.0 ms	0
goji	3650.0 ms	92.4%	1850.0 ms	8000.0 ms	0
klein	3730.0 ms	94.4%	1600.0 ms	7820.0 ms	26
web2py	3950.0 ms	100.0%	2020.0 ms	8000.0 ms	0

Framework overhead of plaintext responses, Dell servers at ServerCentral

Framework best		vs	Platform best		Ratio (higher is better)
fasthttp	2,665,278	vs	go	752,853	354.0%
tornado-pypy2	201,831	vs	tornado	61,738	326.9%
treefrog-hybrid	382,951	vs	treefrog	213,423	179.4%
go-prefork	1,011,589	vs	go	752,853	134.4%
finatra	710,811	vs	finagle	556,888	127.6%
revenj-jvm	884,318	vs	servlet	730,095	121.1%
gemini	824,347	vs	servlet	730,095	112.9%
undertow-jersey-hika	170,106	vs	undertow-jersey-c3p0	154,758	109.9%
treefrog-mongodb	217,082	vs	treefrog	213,423	101.7%
gin	750,780	vs	go	752,853	99.7%
fintrospect	548,336	vs	finagle	556,888	98.5%
vertx-web	2,411,118	vs	vertx	2,503,741	96.3%
actframework	2,424,671	vs	undertow	2,544,955	95.3%
kami	686,682	vs	go	752,853	91.2%
undertow-jersey-c3p0	154,758	vs	undertow-jersey-hika	170,106	91.0%
tornado-py3	55,282	vs	tornado	61,738	89.5%
finch	477,543	vs	finagle	556,888	85.8%
beego	604,659	vs	go	752,853	80.3%
falcon	695,897	vs	wsgi	1,229,303	56.6%
falcon-py3	610,525	vs	wsgi	1,229,303	49.7%
play2-scala	221,514	vs	akka-http	490,038	45.2%
falcore	331,164	vs	go	752,853	44.0%
jetty	305,109	vs	servlet	730,095	41.8%
bottle	462,141	vs	wsgi	1,229,303	37.6%
wicket	271,565	vs	servlet	730,095	37.2%
jetty-servlet	262,519	vs	servlet	730,095	36.0%
bottle-py3	406,822	vs	wsgi	1,229,303	33.1%
jawn	219,577	vs	servlet	730,095	30.1%
weppy	365,421	vs	wsgi	1,229,303	29.7%
weppy-py3	363,672	vs	wsgi	1,229,303	29.6%
pyramid-py2	331,732	vs	wsgi	1,229,303	27.0%
pyramid	310,715	vs	wsgi	1,229,303	25.3%
finagle	556,888	vs	netty	2,447,315	22.8%
tornado-postgresql-r	13,267	vs	tornado	61,738	21.5%
restexpress	497,486	vs	netty	2,447,315	20.3%
echo	150,523	vs	go	752,853	20.0%
tapestry	136,018	vs	servlet	730,095	18.6%
falcon-pypy2	221,939	vs	wsgi	1,229,303	18.1%
spring	131,249	vs	servlet	730,095	18.0%
activeweb	129,324	vs	servlet	730,095	17.7%
curacao	119,759	vs	servlet	730,095	16.4%
revel	117,584	vs	go	752,853	15.6%
play2	70,167	vs	akka-http	490,038	14.3%
django-py3	159,900	vs	wsgi	1,229,303	13.0%
dropwizard	94,713	vs	servlet	730,095	13.0%
jooby	294,958	vs	netty	2,447,315	12.1%
flask-py3	121,901	vs	wsgi	1,229,303	9.9%
flask	113,578	vs	wsgi	1,229,303	9.2%
play2-java-netty	223,814	vs	netty	2,447,315	9.1%
bottle-pypy2	95,722	vs	wsgi	1,229,303	7.8%
play2-scala-scala-ne	154,544	vs	netty	2,447,315	6.3%
django	77,134	vs	wsgi	1,229,303	6.3%
aah	44,783	vs	go	752,853	5.9%
ninja-standalone	13,069	vs	jetty	305,109	4.3%
web2py-optimized	50,373	vs	wsgi	1,229,303	4.1%
goji	22,435	vs	go	752,853	3.0%
weppy-pypy2	22,764	vs	wsgi	1,229,303	1.9%
web2py	20,143	vs	wsgi	1,229,303	1.6%
scruffy	41,366	vs	undertow	2,544,955	1.6%
flask-pypy2	11,316	vs	wsgi	1,229,303	0.9%
flask-nginx-uwsgi	4,868	vs	wsgi	1,229,303	0.4%
weppy-nginx-uwsgi	4,752	vs	wsgi	1,229,303	0.4%
bottle-nginx-uwsgi	4,735	vs	wsgi	1,229,303	0.4%
spark	2,604	vs	servlet	730,095	0.4%

Best (bar chart)

Data table

Latency

Framework overhead

Best (bar chart)

Data table

Latency

Framework overhead

Requirements summary

In this test, the framework responds with the simplest of responses: a "Hello, World" message rendered as plain text. The size of the response is kept small so that gigabit Ethernet is not the limiting factor for all implementations. HTTP pipelining is enabled and higher client-side concurrency levels are used for this test (see the "Data table" view).

Example response:

HTTP/1.1 200 OK Content-Length: 15 Content-Type: text/plain; charset=UTF-8 Server: Example Date: Wed, 17 Apr 2013 12:00:00 GMT Hello, World!

For a more detailed description of the requirements, see the Source Code and Requirements section.

Comments

If you have any comments about Round 15, please post at the Framework Benchmarks Google Group.

Framework details

Author / Sponsor

Home

Framework Source

Test Source

About

JSON

Single query

Multiple query

Fortunes

Updates

Plaintext

About

JSON

Single query

Multiple query

Fortunes

Updates

Plaintext

Results testing

Loading results for run-id from the TFB Results Dashboard...

Unable to load results for run-id from the TFB Results Dashboard.

Running these benchmarks in your own test environment? You can visualize the results by copying and pasting the contents of your results.json file in the text box below.

Test duration: seconds

Visualize results

Limitations

Note: This results web site can only render results for the frameworks it knows about by way of the metadata available for the currently-public Round. Therefore, results for frameworks that are not known by the metadata will not be visible.

Introduction

This is a performance comparison of many web application frameworks executing fundamental tasks such as JSON serialization, database access, and server-side template composition. Each framework is operating in a realistic production configuration. Results are captured on cloud instances and on physical hardware. The test implementations are largely community-contributed and all source is available at the GitHub repository.

Note: We use the word "framework" loosely to refer to platforms, micro-frameworks, and full-stack frameworks.

In a March 2013 blog entry, we published the results of comparing the performance of several web application frameworks executing simple but representative tasks: serializing JSON objects and querying databases. Since then, community input has been tremendous. We—speaking now for all contributors to the project—have been regularly updating the test implementations, expanding coverage, and capturing results in semi-regular updates that we call "rounds."

Results

Ready to see the results of the latest round?

Round 15

Results

View the latest results from Round 15. Or check out the previous rounds.

Making improvements

We expect that all frameworks' tests could be improved with community input. For that reason, we are extremely happy to receive pull requests from fans of any framework. We would like our tests for every framework to perform optimally, so we invite you to please join in.

What's to come

Feedback has been continuous and we plan to keep updating the project in several ways, such as:

Coverage of more frameworks. Thanks to community contributions to-date, the number of frameworks covered has already grown quite large. We're happy to add more if you submit a pull request.
Additional test types.
Tests on more types of hardware.
Enhancements to this results web site.

Additional resources

FAQ	Frequently asked questions and the story behind why we do this.
GitHub	Code for the project at GitHub. Pull Requests welcomed!
Google Group	Mailing list for questions, comments, recommendations, criticisms, or any other form of feedback.
@TFBenchmarks	Tweet at us or follow us on Twitter for quick updates.
TFB-Status	Continuous Benchmarking results (most useful for contributors).

Current and Previous Rounds

Round 15 — The project exceeded 3,000 stars on GitHub and has processed nearly 2,500 pull requests. Continuous benchmarking results are now available on the Results dashboard.

Round 14 — Adoption of the mention-bot from Facebook has proven useful in notifying project participants of changes to their contributions. Continuous benchmarking provided a means for several community previews in this round, and we expect that to continue going forward. Note that this round was conducted only on physical hardware within the ServerCentral environment; tests on the cloud environment will return for Round 15.

Round 13 — Microsoft's ASP.NET team delivers the most impressive improvement we've seen in this project—a 85,000% increase in plaintext results for ASP.NET Core—making it a top-performing framework at the fundamentals of HTTP request routing. Round 13 also sees new hardware and cloud environments from ServerCentral and Microsoft Azure.

Round 12 — Marking the last round on the Peak environment, Round 12 sees some especially high Plaintext scores.

Round 11 — 26 more frameworks, three more languages, and the volume cranked to 11.

Round 10 — Significant restructuring of the project's infrastructure, including re-organization of the project's directory structure and integration with Travis CI for rapid review of pull requests, and the addition of numerous frameworks.

Round 9 — Thanks to the contribution of a 10-gigabit testing environment by Peak Hosting, the network barrier that frustrated top-performing frameworks in previous rounds has been removed. The Dell R720xd servers in this new environment feature dual Xeon E5-2660 v2 processors and illustrate how the spectrum of frameworks scale to forty processor cores.

Round 8 — Six more frameworks contributed by the community takes the total count to 90 frameworks and 230 permutations (variations of configuration). Meanwhile, several implementations have been updated and the highest-performance platforms jockey for the top spot on each test's charts.

Round 7 — After a several month hiatus, another large batch of frameworks have been added by the community. Even after consolidating a few, Round 7 counts 84 frameworks and over 200 test permutations! This round also was the first to use a community-review process. Future rounds will see roughly one week of preview and review by the community prior to release to the public here.

Round 6 — Still more tests were contributed by the developer community, bringing the number of frameworks to 74! Round 6 also introduces an "plaintext" test type that exercises HTTP pipelining and higher client-side concurrency levels.

Round 5 — The developer community comes through with the addition of ASP.NET tests ready to run on Windows. This round is the first with Windows tests, and we seek assistance from Windows experts to apply additional tuning to bring the results to parity with the Linux tests. Round 5 also introduces an "update" test type to exercise ORM and database writes.

Round 4 — With 57 frameworks in the benchmark suite, we've added a filter control allowing you to narrow your view to only the frameworks you want to see. Round 4 also introduces the "Fortune" test to exercise server-side templates and collections.

Round 3 — We created this stand-alone site for comparing the results data captured across many web application frameworks. Even more frameworks have been contributed by the community and the testing methodology was changed slightly thanks to enhancements to the testing tool named Wrk.

Round 2 — In April, we published a follow-up blog entry named "Frameworks Round 2" where we incorporated changes suggested and contributed by the community.

Round 1 — In a March 2013 blog entry, we published the results of comparing the performance of several web application frameworks executing simple but representative tasks: serializing JSON objects and querying databases. The community reaction was terrific. We are flattered by the volume of feedback. We received dozens of comments, suggestions, questions, criticisms, and most importantly, GitHub pull requests at the repository we set up for this project.

Unofficial Results

We operate a continuously-running benchmarking environment. You can see unofficial results as they are collected at the TFB Results Dashboard.

Motivation

Choosing a web application framework involves evaluation of many factors. While comparatively easy to measure, performance is frequently given little consideration. We hope to help change that.

Application performance can be directly mapped to hosting dollars, and for companies both large and small, hosting costs can be a pain point. Weak performance can also cause premature and costly scale pain by requiring earlier optimization efforts and increased architectural complexity. Finally, slow applications yield poor user experience and may suffer penalties levied by search engines.

What if building an application on one framework meant that at the very best your hardware is suitable for one tenth as much load as it would be had you chosen a different framework? The differences aren't always that extreme, but in some cases, they might be. Especially with several modern high-performance frameworks offering respectable developer efficiency, it's worth knowing what you're getting into.

Terminology

framework: We use the word framework loosely to refer to any HTTP server implementation upon which you could build a web application—a full-stack framework, a micro-framework, or even a web platform such as Rack, Servlet, or plain PHP.
platform: For us, platforms are broadly defined as anything situated between the programming language and the web framework (examples are Servlet, Netty, and Rack). By comparison to a full-stack framework or micro-framework, a platform may include a bare-bones HTTP server implementation with rudimentary request routing and virtually none of the higher-order functionality of frameworks such as form validation, input sanitization, templating, JSON serialization, and database connectivity. Frameworks are often built on top of platforms. To be thorough, and to compute framework overhead, we test several platforms as if they were frameworks.
permutation: A combination of attributes that compose a full technology stack being tested (take node.js for example, we might have one permutation with MongoDB and another with MySQL). Some frameworks have seen many permutations contributed by the community; others only one or few.
test type: One of the workloads we exercise, such as JSON serialization, single-query, multiple-query, fortunes, data updates, and plaintext.
test: An individual test is a measurement of the performance of a permutation's implementation of a test type. For example, a test might be measuring Wicket paired with MySQL running the single-query test type.
implementation: Sometimes called "test implementations," these are the bodies of code and configuration created to test permutations according to the requirements. These are frequently contributed by fans, advocates, or the maintainers of frameworks. Together with the toolset, test implementations are the meat of this project.
toolset: A set of Python scripts that run our tests.
run: An execution of the benchmark toolset across the suite of test implementations, either in full or in part, in order to capture results for any purpose.
preview: A capture of data from a run used by project participants to sanity-check prior to an official round.
round: A posting of "official" results on this web site. This is mostly for ease of consumption by readers and good-spirited & healthy competitive bragging rights. For in-depth analysis, we encourage you to examine the source code and run the tests on your own hardware.

Expected questions

We expect that you might have a bunch of questions. Here are some that we're anticipating. But please contact us if you have a question we're not dealing with here or just want to tell us we're doing it wrong.

Frameworks and configuration

"You call x a framework, but it's a platform." See the terminology section above. We are using the word "framework" loosely to refer to anything found on the spectrum ranging from full-stack frameworks, micro-frameworks, to platforms. If it's used to build web applications, it probably qualifies. That said, we understand that comparing a full-stack framework versus platforms or vice-versa is unusual. We feel it's valuable to be able to compare these, for example to understand the performance overhead of additional abstraction. You can use the filters in the results viewer to adjust the rows you see in the charts.
"You configured framework x incorrectly, and that explains the numbers you're seeing." Whoops! Please let us know how we can fix it, or submit a GitHub pull request, so we can get it right.
"Why include this Gemini framework I've never heard of?" We have included our in-house Java web framework, Gemini, in our tests. We've done so because it's of interest to us. You can consider it a stand-in for any relatively lightweight minimal-locking Java framework. While we're proud of how it performs among the well-established field, this exercise is not about Gemini.
"Why don't you test framework X?" We'd love to, if we can find the time. Even better, craft the test implementation yourself and submit a GitHub pull request so we can get it in there faster!
"Some frameworks use process-level concurrency; have you accounted for that?" Yes, we've attempted to use production-grade configuration settings for all frameworks, including those that rely on process-level concurrency. For the EC2 tests, for example, such frameworks are configured to utilize the two virtual cores provided on an c3.large (in previous rounds, m1.large) instance. For the i7 tests, they are configured to use the eight hyper-threading cores of our hardware's i7 CPUs.
"Have you enabled APC for the PHP tests?" Yes, the PHP tests run with APC and PHP-FPM on nginx.
"Why are you using a (slightly) old version of framework X?" It's nothing personal! With so many frameworks we have a never-ending game of whack-a-mole. If you think an update will affect the results, please let us know (or better yet, submit a GitHub pull request) and we'll get it updated!
"It's unfair and possibly even incorrect to compare X and Y!" It may be alarming at first to see the full results table, where one may evaluate frameworks vs platforms; MySQL vs Postgres; Go vs Python; ORM vs raw database connectivity; and any number of other possibly irrational comparisons. Many readers desire the ability to compare these and other permutations. If you prefer to view an unpolluted subset, you may use the filters available at the top of the results page. We believe that comparing frameworks with plausible and diverse technology stacks, despite the number of variables, is precisely the value of this project. With sufficient time and effort, we hope to continuously broaden the test permutations. But we recommend against ignoring the data on the basis of concerns about multi-variable comparisons. Read more opinion on this at Brian Hauer's personal blog.
"If you are testing production deployments, why is logging disabled?" At present, we have elected to run tests with logging features disabled. Although this is not consistent with production deployments, we avoid a few complications related to logging, most notably disk capacity and consistent granularity of logging across all test implementations. In spot tests, we have not observed significant performance impact from logging when enabled. If there is strong community consensus that logging is necessary, we will reconsider this.
"Tell me about the Windows configuration." We are very thankful to the community members who have contributed Windows tests. In fact, nearly the entirety of the Windows configuration has been contributed by subject-matter experts from the community. Thanks to their effort, we now have tests covering both Windows paired with Linux databases and Windows paired with Microsoft SQL Server. As with all aspects of this project, we welcome continued input and tuning by other experts. If you have advice on better tuning the Windows tests, please submit GitHub issues or pull requests.

The tests

"Framework X has in-memory caching, why don't you use that?" In-memory caching, as provided by some frameworks, yields higher performance than repeatedly hitting a database, but isn't available in all frameworks, so we omitted in-memory caching from these tests. Cache tests are planned for later rounds.
"What about other caching approaches, then?" Remote-memory or near-memory caching, as provided by Memcached and similar solutions, also improves performance and we would like to conduct future tests simulating a more expensive query operation versus Memcached. However, curiously, in spot tests, some frameworks paired with Memcached were conspicuously slower than other frameworks directly querying the authoritative MySQL database (recognizing, of course, that MySQL had its entire data-set in its own memory cache). For simple "get row ID n" and "get all rows" style fetches, a fast framework paired with MySQL may be faster and easier to work with versus a slow framework paired with Memcached.
"Why doesn't your test include more substantial algorithmic work?" Great suggestion. We hope to in the future!
"What about reverse proxy options such as Varnish?" We are expressly not using reverse proxies on this project. There are other benchmark projects that evaluate the performance of reverse proxy software. This project measures the performance of web applications in any scenario where requests reach the application server. Given that objective, allowing the web application to avoid doing the work thanks to a reverse proxy would invalidate the results. If it's difficult to conceptualize the value of measuring performance beyond the reverse proxy, imagine a scenario where every response provides user-specific and varying data. It's also notable that some platforms respond with sufficient performance to potentially render a reverse proxy unnecessary.
"Do all the database tests use connection pooling?" Yes, our expectation is that all tests use connection pooling.
"How is each test run?" Each test is executed as follows:
1. Restart the database servers.
2. Start the platform and framework using their start-up mechanisms.
3. Run a 5-second primer at 8 client-concurrency to verify that the server is in fact running. These results are not captured.
4. Run a 15-second warmup at 256 client-concurrency to allow lazy-initialization to execute and just-in-time compilation to run. These results are not captured.
5. Run a 15-second captured test for each of the concurrency levels (or iteration counts) exercised by the test type. Concurrency-variable test types are tested at 8, 16, 32, 64, 128, and 256 client-side concurrency. The high-concurrency plaintext test type is tested at 256, 1,024, 4,096, and 16,384 client-side concurrency.
6. Stop the platform and framework.
"Hold on, 15 seconds is not enough to gather useful data." This is a reasonable concern. But in examining the data, we have seen no evidence that the results have changed by reducing the individual test durations from 60 seconds to 15 seconds. The duration reduction was made necessary by the growing number of test permutations and a target that the full suite complete in less than one day. With additional effort, we aim to build a continuously-running test environment that will pull the latest source and begin a new run as soon as a previous run completes. When we have such an environment ready, we will be comfortable with multi-day execution times, so we plan to extend the duration of each test when that happens.
"Also, a 15-second warmup is not sufficient." On the contrary, we have not yet seen evidence suggesting that any additional warmup time is beneficial to any framework. In fact, for frameworks based on JIT platforms such as the Java Virtual Machine (JVM), spot tests show that the JIT has even completed its work already after just the primer and before the warmup starts—the warmup (256-concurrency) and real 256-concurrency tests yield results that are separated only by test noise. However, as with test durations, we intend to increase the duration of the warmup when we have a continuously-running test environment.

Environment

"What is Wrk?" Although many web performance tests use ApacheBench from Apache to generate HTTP requests, we now use Wrk for this project. ApacheBench remains a single-threaded tool, meaning that for higher-performance test scenarios, ApacheBench itself is a limiting factor. Wrk is a multithreaded tool that provides a similar function, allowing tests to run for a prescribed amount of time (rather than limited to a number of requests) and providing us result data including total requests completed and latency information.
"Doesn't benchmarking on Amazon EC2 invalidate the results?" Our opinion is that doing so confirms precisely what we're trying to test: performance of web applications within realistic production environments. Selecting EC2 as a platform also allows the tests to be readily verified by anyone interested in doing so. However, we've also executed tests on our Core i7 (Sandy Bridge) workstations running Ubuntu as a non-virtualized comparison. Doing so confirmed our suspicion that the ranked order and relative performance across frameworks is mostly consistent between EC2 and physical hardware. That is, while the EC2 instances were slower than the physical hardware, they were slower by roughly the same proportion across the spectrum of frameworks.
"Tell me about your physical hardware." For the tests we refer to as "i7" tests, we're using our office workstations. These use Intel i7-2600K processors, making them a little antiquated, to be honest. These are connected via an unmanaged low-cost gigabit Ethernet switch. In previous rounds, we used a two-machine configuration where the load-generation and database role coexisted. Although these two roles were not crowding one another out (neither role was starved for CPU time), as of Round 7, we are using a three-machine configuration for the physical hardware tests. The machine roles are:
- Application server, which hosts the application code and web server, where applicable.
- Database server, which hosts the common databases. Starting with Round 5, we equipped the database server with a Samsung 840 Pro SSD.
- Load generator, which makes HTTP requests to the Application server via the Wrk load generation tool.
"What is Resin? Why aren't you using Tomcat for the Java frameworks?" Resin is a Java application server. The GPL version that we used for our tests is a relatively lightweight Servlet container. We tested on Tomcat as well but ultimately dropped Tomcat from our tests because Resin was slightly faster across all Servlet-based frameworks.
"Do you run any warmups before collecting results data?" Yes. See "how is each test run" above. Every test is preceded by a warmup and brief (several seconds) cooldown prior to gathering test data.

Results

"I am about to start a new web application project; how should I interpret these results?" Most importantly, recognize that performance data should be one part of your decision-making process. High-performance web applications reduce hosting costs and improve user experience. Additionally, recognize that while we have aimed to select test types that represent workloads that are common for web applications, nothing beats conducting performance tests yourself for the specific workload of your application. In addition to performance, consider other requirements such as your language and platform preference; your invested knowledge in one or more of the frameworks we've tested; and the documentation and support provided by the framework's community. Combined with an examination of the source code, the results seen here should help you identify a platform and framework that is high-performance while still meeting your other requirements.
"Why are the leaderboards for JSON Serialization and Plaintext so different on EC2 versus i7?" Put briefly, for fast frameworks on our i7 physical hardware, the limiting factor for the JSON test is our gigabit Ethernet; whereas on EC2, the limit is the CPU. Assuming proper response headers are provided, at approximately 200,000 non-pipelined and 550,000 pipelined responses per second and above, the network is saturated.
"Where did earlier rounds go?" To better capture HTTP errors reported by Wrk, we have restructured the format of our results.json file. The test tool changed at Round 2 and some framework IDs were changed at Round 3. As a result, the results.json for Rounds 1 and 2 would have required manual editing and we opted to simply remove the previous rounds from this site. You can still see those rounds at our blog: Round 1, Round 2.
"What does 'Did not complete' mean?" Starting with Round 9, we have added validation checks to confirm that implementations are behaving as we have specified in the requirements section of this site. An implementation that does not return the correct results, bypasses some of the requirements, or even formats the results in a manner inconsistent with the requirements will be marked as "Did not complete." We have solicited corrections from prior contributors and have attempted to address many of these, but it will take more time for all implementations to be correct. If you are a project participant and your contribution is marked as "Did not complete," please help us resolve this by contacting us at the GitHub repository. We may ultimately need a pull request from you, but we'd be happy to help you understand what specifically is triggering a validation error with your implementation.
"Why are Stripped test implementations hidden by default?" Since the introduction of Stripped test implementations, we have debated whether they should be included at all. A Stripped test implementation is one that is specially crafted to excel at our benchmark. By comparison, a "Realistic" test implementation should be demonstrative of the general-purpose, best-practices compliant, and production-class approach for the given framework. We have decided to hide Stripped tests by default because we feel that while their results have some value, that value is exceedingly low for the vast majority of consumers of the data. You may still view the results for Stripped tests by enabling the Stripped Implementation Approach in the filters control panel.
"What exactly causes a test implementation to be classified as Stripped?" It's not possible to paint an exact picture, but conceptually, a Stripped implementation is characterized by being configured or engineered expressly to the requirements of our benchmark tests. By comparison, a Realistic implementation will use a production-grade configuration of a general-purpose web application framework that meets our requirements. When we first introduced the notion of Stripped, it referred to configurations of otherwise normal software that had been stripped of some normal behaviors. For example, removing some middleware from Rails or Django. However, we have broadened the definition of Stripped to also refer to bespoke software that has been crafted expressly to meet our test types' requirements.
"I have collected results from my own test environment; can I visualize them in a manner similar to this web site?" Yes, use the test results visualization feature to visualize results you've gathered. Be aware that you need to specify the per-test duration setting (in seconds) and that only known frameworks will be rendered.

Contributions

"Do you accept contributions?" Absolutely! Please visit the project's GitHub repository to join the project. In fact, the majority of the test implementations are community-contributed.
"You emphasize production-grade software. Do you accept early builds of frameworks or toy projects?" Actually, yes, we are quite liberal with accepting contributions, including those that don't meet the target of production-grade. However, in many cases, such implementations will be marked as "Stripped," meaning they are not recommended for use in real-world production projects. They will still be measured and can be made visible by enabling Stripped in the filters control panel.
"I am a contributor and my framework hasn't shown up in an official round yet. How do I know how my contribution will performn in your test environment?" We have a benchmark environment that is continuously running the benchmark suite, which we imaginatively named "Continuous Benchmarking™" (not actually trademarked). The continuous benchmark runner posts results to the TFB Results Dashboard. If you have made a contribution, you should see its results posted to the dashboard within a few days to a week, when the next regular continuous run completes.

Join the conversation

FAQ	Frequently asked questions and the story behind why we do this.
GitHub	Code for the project at GitHub. Pull Requests welcomed!
Google Group	Mailing list for questions, comments, recommendations, criticisms, or any other form of feedback.
@TFBenchmarks	Tweet at us or follow us on Twitter for quick updates.
TFB-Status	Continuous Benchmarking results (most useful for contributors).

FAQ	Frequently asked questions and the story behind why we do this.
GitHub	Code for the project at GitHub. Pull Requests welcomed!
Google Group	Mailing list for questions, comments, recommendations, criticisms, or any other form of feedback.
@TFBenchmarks	Tweet at us or follow us on Twitter for quick updates.
TFB-Status	Continuous Benchmarking results (most useful for contributors).

FAQ	Frequently asked questions and the story behind why we do this.
GitHub	Code for the project at GitHub. Pull Requests welcomed!
Google Group	Mailing list for questions, comments, recommendations, criticisms, or any other form of feedback.
@TFBenchmarks	Tweet at us or follow us on Twitter for quick updates.
TFB-Status	Continuous Benchmarking results (most useful for contributors).

FAQ	Frequently asked questions and the story behind why we do this.
GitHub	Code for the project at GitHub. Pull Requests welcomed!
Google Group	Mailing list for questions, comments, recommendations, criticisms, or any other form of feedback.
@TFBenchmarks	Tweet at us or follow us on Twitter for quick updates.
TFB-Status	Continuous Benchmarking results (most useful for contributors).

Simulating production environments

We aim to configure every framework according to the best practices for production deployments gleaned from documentation and popular community opinion, and we ask contributors to apply the same rule of thumb. We want each test implementation (see "Terminology" section) to approximate a sensible production deployment as accurately as possible. We also want this project to be as transparent as possible, so we have posted our test suites on GitHub.

Environment details

This project measures performance in two common deployment scenarios: cloud instances and physical hardware. To-date, each round has used a single representative environment for each of these scenarios. The particular specifications of the environments have evolved over time as shown below.

Cloud environments

Azure (rounds 13 onward): Microsoft Azure D3v2 instances; switched gigabit Ethernet.
AWS (rounds 1 through 12): Amazon EC2 c3.large instances (2 vCPU each); switched gigabit Ethernet (m1.large was used through Round 9).

Physical hardware environments

Citrine (rounds 16 onward): Three homogeneous Dell R440 servers each equipped with an Intel Xeon Gold 5120 CPU, 32 GB of memory, and an enterprise SSD. Dedicated Cisco 10-gigabit Ethernet switch. Provided by Microsoft.
ServerCentral (rounds 13 through 15): Dell R910 (4x 10-Core Intel Xeon E7-4850 CPUs) application server; Dell R710 (2x 4-Core Intel Xeon E5520 CPUs) database server; switched 10-gigabit Ethernet. Provided by ServerCentral.
Peak (rounds 9 through 12): Dell R720xd dual Intel Xeon E5-2660 v2 (40 HT cores) with 32 GB memory; database servers equipped with SSDs in RAID; switched 10-gigabit Ethernet. Provided by Peak Hosting.
i7 (rounds 1 through 8): In-house Intel Sandy Bridge Core i7-2600K workstations with 8 GB memory (early 2011 vintage); database server equipped with Samsung 840 Pro SSD; switched gigabit Ethernet.

General resources

If you are interested in this project, we'd love to hear from you.

FAQ	Frequently asked questions and the story behind why we do this.
GitHub	Code for the project at GitHub. Pull Requests welcomed!
Google Group	Mailing list for questions, comments, recommendations, criticisms, or any other form of feedback.
@TFBenchmarks	Tweet at us or follow us on Twitter for quick updates.
TFB-Status	Continuous Benchmarking results (most useful for contributors).

FAQ	Frequently asked questions and the story behind why we do this.
GitHub	Code for the project at GitHub. Pull Requests welcomed!
Google Group	Mailing list for questions, comments, recommendations, criticisms, or any other form of feedback.
@TFBenchmarks	Tweet at us or follow us on Twitter for quick updates.
TFB-Status	Continuous Benchmarking results (most useful for contributors).

FAQ	Frequently asked questions and the story behind why we do this.
GitHub	Code for the project at GitHub. Pull Requests welcomed!
Google Group	Mailing list for questions, comments, recommendations, criticisms, or any other form of feedback.
@TFBenchmarks	Tweet at us or follow us on Twitter for quick updates.
TFB-Status	Continuous Benchmarking results (most useful for contributors).

FAQ	Frequently asked questions and the story behind why we do this.
GitHub	Code for the project at GitHub. Pull Requests welcomed!
Google Group	Mailing list for questions, comments, recommendations, criticisms, or any other form of feedback.
@TFBenchmarks	Tweet at us or follow us on Twitter for quick updates.
TFB-Status	Continuous Benchmarking results (most useful for contributors).

Test requirements

We invite fans of frameworks and especially authors or maintainers of frameworks to join us in expanding the coverage of this project by implementing tests and contributing to the GitHub repository. The following are specifications for each of the test types we have included to-date in this project. Do not be alarmed; the specifications read quite verbose, but that's because they are specifications. The implementations tend to be quite easy in practice.

This project is evolving and we will periodically add new test types. As new test types are added, we encourage but do not require contributors of previous implementations to implement tests for the new test types. Wholly new test implementations are also encouraged to include all test types but are not required to do so. If you have limited time, we recommend you start with the easiest test types (1, 2, 3, and 6) and then continue beyond those as time permits.

General requirements

The following requirements apply to all test types below.

All test implementations should be production-grade. The particulars of this will vary by framework and platform, but the general sentiment is that the code and configuration should be suitable for a production deployment. The word should is used here because production-grade is our goal, but we don't want this to be a roadblock. If you're submitting a new test and uncertain whether your code is production-grade, submit it anyway and then solicit input from other subject-matter experts.
This project intends to measure performance of production-grade deployments of real-world web application frameworks and platforms. All implementations are expected (but not required) to be based on robust implementations of the HTTP protocol. Implementations that are not based on a realistic HTTP implementation will be marked as Stripped (see Questions section for more detail on the Stripped classification). Rather than stipulate a rigid specification of HTTP requirements, we defer to common sense: the implementation should provide a viable subset of HTTP to reasonably unbiased outsiders familiar with HTTP.
All test implementations must disable all disk logging. For many reasons, we expect all tests will run without writing logs to disk. Most importantly, the volume of requests is sufficiently high to fill up disks even with only a single line written to disk per request. Please disable all forms of disk logging. We recommend but do not require disabling console logging as well.
Specific characters and character case matter. Assume the client consuming your service's JSON responses will be using a case-sensitive language such as JavaScript. In other words, if a test specifies that a map's key is id, use id. Do not use Id or ID. This strictness is required not only because it's sensible but also because our automated validation checks are picky.
All test types require Server and Date HTTP response headers. We expect the Server header to be whatever is normal for the platform or framework. If the framework does not normally provide a Server response header, we nevertheless require that you provide one as this roughly normalizes network load across all implementations. For Date, we expect that the rendered date be accurate. However, it does not need to be rendered from the system clock to a byte buffer for each request. Re-rendering once per second is an acceptable tactical optimization (and is an optimization baked into many frameworks).
Some form of request routing is required, even if only a single test type (e.g., Plaintext) is implemented. In most cases, the framework's canonical router or a mainstream router library will be used. In some cases, it is considered normal and sufficiently production-grade to use hand-crafted minimalist routing using control structures such as if/else branching. This is acceptable where it is considered canonical for the framework.
Except where noted, all database queries should be delivered to the database servers as-is and not coalesced or deduplicated at the database driver. In all cases where a database query is required, it is expected that the query will reach and execute on the database server—that is, the full round-trip effort of a query will be part of the workload.

Test type 1: JSON serialization

This test exercises the framework fundamentals including keep-alive support, request routing, request header parsing, object instantiation, JSON serialization, response header generation, and request count throughput.

Requirements

For each request, an object mapping the key message to Hello, World! must be instantiated.
The recommended URI is /json.
A JSON serializer must be used to convert the object to JSON.
The response text must be {"message":"Hello, World!"}, but white-space variations are acceptable.
The response content length should be approximately 28 bytes.
The response content type must be set to application/json. Note that specifying the character encoding is unnecessary for this content type.
The response headers must include either Content-Length or Transfer-Encoding.
The response headers must include Server and Date.
gzip compression is not permitted.
Server support for HTTP Keep-Alive is strongly encouraged but not required.
If HTTP Keep-Alive is enabled, no maximum Keep-Alive timeout is specified by this test.
The request handler will be exercised at concurrency levels ranging from 8 to 256.
The request handler will be exercised using GET requests.

Example request

GET /json HTTP/1.1 Host: server User-Agent: Mozilla/5.0 (X11; Linux x86_64) Gecko/20130501 Firefox/30.0 AppleWebKit/600.00 Chrome/30.0.0000.0 Trident/10.0 Safari/600.00 Cookie: uid=12345678901234567890; __utma=1.1234567890.1234567890.1234567890.1234567890.12; wd=2560x1600 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-US,en;q=0.5 Connection: keep-alive

Example response

HTTP/1.1 200 OK Content-Type: application/json Content-Length: 28 Server: Example Date: Wed, 17 Apr 2013 12:00:00 GMT {"message":"Hello, World!"}

Test type 2: Single database query

This test exercises the framework's object-relational mapper (ORM), random number generator, database driver, and database connection pool.

Requirements

For every request, a single row from a World table must be retrieved from a database table.
The recommended URI is /db.
The schema for World is id (int, primary key) and randomNumber (int), except for MongoDB, wherein the identity column is _id, with the leading underscore.
The World table is known to contain 10,000 rows.
The row retrieved must be selected by its id using a random number generator (ids range from 1 to 10,000).
The row should be converted to an object using an object-relational mapping (ORM) tool. Tests that do not use an ORM will be classified as "raw" meaning they use the platform's raw database connectivity.
The object (or database row, if an ORM is not used) must be serialized to JSON.
The response content length should be approximately 32 bytes.
The response content type must be set to application/json. Note that specifying the character encoding is unnecessary for this content type.
The response headers must include either Content-Length or Transfer-Encoding.
The response headers must include Server and Date.
Use of an in-memory cache of World objects or rows by the application is not permitted.
Use of prepared statements for SQL database tests (e.g., for MySQL) is encouraged but not required.
gzip compression is not permitted.
Server support for HTTP Keep-Alive is strongly encouraged but not required.
If HTTP Keep-Alive is enabled, no maximum Keep-Alive timeout is specified by this test.
The request handler will be exercised at concurrency levels ranging from 8 to 256.
The request handler will be exercised using GET requests.

Example request

GET /db HTTP/1.1 Host: server User-Agent: Mozilla/5.0 (X11; Linux x86_64) Gecko/20130501 Firefox/30.0 AppleWebKit/600.00 Chrome/30.0.0000.0 Trident/10.0 Safari/600.00 Cookie: uid=12345678901234567890; __utma=1.1234567890.1234567890.1234567890.1234567890.12; wd=2560x1600 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-US,en;q=0.5 Connection: keep-alive

Example response

HTTP/1.1 200 OK Content-Length: 32 Content-Type: application/json Server: Example Date: Wed, 17 Apr 2013 12:00:00 GMT {"id":3217,"randomNumber":2149}

Test type 3: Multiple database queries

This test is a variation of Test #2 and also uses the World table. Multiple rows are fetched to more dramatically punish the database driver and connection pool. At the highest queries-per-request tested (20), this test demonstrates all frameworks' convergence toward zero requests-per-second as database activity increases.

Requirements

For every request, an integer query string parameter named queries must be retrieved from the request. The parameter specifies the number of database queries to execute in preparing the HTTP response (see below).
The recommended URI is /queries.
The queries parameter must be bounded to between 1 and 500. If the parameter is missing, is not an integer, or is an integer less than 1, the value should be interpreted as 1; if greater than 500, the value should be interpreted as 500.
The request handler must retrieve a set of World objects, equal in count to the queries parameter, from the World database table.
Each row must be selected randomly in the same fashion as the single database query test (Test #2 above).
This test is designed to exercise multiple queries, each requiring a round-trip to the database server, and with each resulting row selected individually. It is not acceptable to use batches. It is not acceptable to execute multiple SELECTs within a single statement. It is not acceptable to retrieve all required rows using a SELECT ... WHERE id IN (...) clause.
Each World object must be added to a list or array.
The list or array must be serialized to JSON and sent as a response.
The response content type must be set to application/json. Note that specifying the character encoding is unnecessary for this content type.
The response headers must include either Content-Length or Transfer-Encoding.
The response headers must include Server and Date.
Use of an in-memory cache of World objects or rows by the application is not permitted.
Use of prepared statements for SQL database tests (e.g., for MySQL) is encouraged but not required.
gzip compression is not permitted.
Server support for HTTP Keep-Alive is strongly encouraged but not required.
If HTTP Keep-Alive is enabled, no maximum Keep-Alive timeout is specified by this test.
The request handler will be exercised at 256 concurrency only.
The request handler will be exercised with query counts of 1, 5, 10, 15, and 20.
The request handler will be exercised using GET requests.

Example request

GET /queries?queries=10 HTTP/1.1 Host: server User-Agent: Mozilla/5.0 (X11; Linux x86_64) Gecko/20130501 Firefox/30.0 AppleWebKit/600.00 Chrome/30.0.0000.0 Trident/10.0 Safari/600.00 Cookie: uid=12345678901234567890; __utma=1.1234567890.1234567890.1234567890.1234567890.12; wd=2560x1600 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-US,en;q=0.5 Connection: keep-alive

Example response

Test type 4: Fortunes

This test exercises the ORM, database connectivity, dynamic-size collections, sorting, server-side templates, XSS countermeasures, and character encoding.

Requirements

The recommended URI is /fortunes.
A Fortune database table contains a dozen Unix-style fortune-cookie messages.
The schema for Fortune is id (int, primary key) and message (varchar), except for MongoDB, wherein the identity column is _id, with the leading underscore.
Using an ORM, all Fortune objects must be fetched from the Fortune table, and placed into a list data structure. Tests that do not use an ORM will be classified as "raw" meaning they use the platform's raw database connectivity.
Each request to the Fortunes URI should result in exactly one database query as specified above. Query coalescing or deduplication is not permitted. See the General requirements section above. Conceptually, the expectation is that after measuring an implementation of the Fortunes test type, the aggregate number of queries executed by the database server during the test is equal to the number of Fortunes HTTP requests completed.
The list data structure must be a dynamic-size or equivalent and should not be dimensioned using foreknowledge of the row-count of the database table.
Within the scope of the request, a new Fortune object must be constructed and added to the list. This confirms that the data structure is dynamic-sized. The new fortune is not persisted to the database; it is ephemeral for the scope of the request.
The new Fortune's message must be "Additional fortune added at request time."
The list of Fortune objects must be sorted by the order of the message field. No ORDER BY clause is permitted in the database query (ordering within the query would be of negligible value anyway since a newly instantiated Fortune is added to the list prior to sorting).
The sorted list must be provided to a server-side template and rendered to simple HTML (see below for minimum template). The resulting HTML table displays each Fortune's id number and message text.
This test does not include external assets (CSS, JavaScript); a later test type will include assets.
The HTML generated by the template must be sent as a response.
Be aware that the message text fields are stored as UTF-8 and one of the fortune cookie messages is in Japanese.
The resulting HTML must be delivered using UTF-8 encoding.
The Japanese fortune cookie message must be displayed correctly.
Be aware that at least one of the message text fields includes a <script> tag.
The server-side template must assume the message text cannot be trusted and must escape the message text properly.
The implementation is encouraged to use best practices for templates such as layout inheritence, separate header and footer files, and so on. However, this is not required. We request that implementations do not manage assets (JavaScript, CSS, images). We are deferring asset management until we can craft a more suitable test.
The response content type must be set to text/html.
The response headers must include either Content-Length or Transfer-Encoding.
The response headers must include Server and Date.
Use of an in-memory cache of Fortune objects or rows by the application is not permitted.
Use of prepared statements for SQL database tests (e.g., for MySQL) is encouraged but not required.
gzip compression is not permitted.
Server support for HTTP Keep-Alive is strongly encouraged but not required.
If HTTP Keep-Alive is enabled, no maximum Keep-Alive timeout is specified by this test.
The request handler will be exercised at concurrency levels ranging from 8 to 256.
The request handler will be exercised using GET requests.

Example request

GET /fortunes HTTP/1.1 Host: server User-Agent: Mozilla/5.0 (X11; Linux x86_64) Gecko/20130501 Firefox/30.0 AppleWebKit/600.00 Chrome/30.0.0000.0 Trident/10.0 Safari/600.00 Cookie: uid=12345678901234567890; __utma=1.1234567890.1234567890.1234567890.1234567890.12; wd=2560x1600 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-US,en;q=0.5 Connection: keep-alive

Example response

Minimum template

Along with the example response above, the following Mustache template illustrates the minimum requirements for the server-side template. White-space can be optionally eliminated.

<!DOCTYPE html> <html> <head><title>Fortunes</title></head> <body> <table> <tr><th>id</th><th>message</th></tr> {{#.}} <tr><td>{{id}}</td><td>{{message}}</td></tr> {{/.}} </table> </body> </html>

Test type 5: Database updates

This test is a variation of Test #3 that exercises the ORM's persistence of objects and the database driver's performance at running UPDATE statements or similar. The spirit of this test is to exercise a variable number of read-then-write style database operations.

Requirements

The recommended URI is /updates.
For every request, an integer query string parameter named queries must be retrieved from the request. The parameter specifies the number of rows to fetch and update in preparing the HTTP response (see below).
The queries parameter must be bounded to between 1 and 500. If the parameter is missing, is not an integer, or is an integer less than 1, the value should be interpreted as 1; if greater than 500, the value should be interpreted as 500.
The request handler must retrieve a set of World objects, equal in count to the queries parameter, from the World database table.
Each row must be selected randomly using one query in the same fashion as the single database query test (Test #2 above). As with the read-only multiple-query test type (#3 above), use of IN clauses or similar means to consolidate multiple queries into one operation is not permitted. Similarly, use of a batch or multiple SELECTs within a single statement are not permitted.
At least the randomNumber field must be read from the database result set.
Each World object must have its randomNumber field updated to a new random integer between 1 and 10000.
Each World object must be persisted to the database with its new randomNumber value.
Use of batch updates is acceptable but not required. To be clear: batches are not permissible for selecting/reading the rows, but batches are acceptable for writing the updates.
Use of transactions is acceptable but not required. If transactions are used, a transaction should only encapsulate a single iteration, composed of a single read and single write. Transactions should not be used to consolidate multiple iterations into a single operation.
All updates should be completed prior to returning the HTTP response.
For raw tests (that is, tests without an ORM), each updated row must receive a unique new randomNumber value. It is not acceptable to change the randomNumber value of all rows to the same random number using an UPDATE ... WHERE id IN (...) clause.
Each World object must be added to a list or array.
The list or array must be serialized to JSON and sent as a response.
The response content type must be set to application/json. Note that specifying the character encoding is unnecessary for this content type.
The response headers must include either Content-Length or Transfer-Encoding.
The response headers must include Server and Date.
Use of an in-memory cache of World objects or rows by the application is not permitted.
Use of prepared statements for SQL database tests (e.g., for MySQL) is encouraged but not required.
gzip compression is not permitted.
Server support for HTTP Keep-Alive is strongly encouraged but not required.
If HTTP Keep-Alive is enabled, no maximum Keep-Alive timeout is specified by this test.
The request handler will be exercised at 256 concurrency only.
The request handler will be exercised with query counts of 1, 5, 10, 15, and 20.
The request handler will be exercised using GET requests.

Example request

GET /updates?queries=10 HTTP/1.1 Host: server User-Agent: Mozilla/5.0 (X11; Linux x86_64) Gecko/20130501 Firefox/30.0 AppleWebKit/600.00 Chrome/30.0.0000.0 Trident/10.0 Safari/600.00 Cookie: uid=12345678901234567890; __utma=1.1234567890.1234567890.1234567890.1234567890.12; wd=2560x1600 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-US,en;q=0.5 Connection: keep-alive

Example response

Test type 6: Plaintext

This test is an exercise of the request-routing fundamentals only, designed to demonstrate the capacity of high-performance platforms in particular. Requests will be sent using HTTP pipelining. The response payload is still small, meaning good performance is still necessary in order to saturate the gigabit Ethernet of the test environment.

The General Requirements section above applies to all test types, including Plaintext. Importantly, implementations should be production-grade deployments of real-world web application frameworks or platforms. The Plaintext test type is historically highly competitive, leading to some implementations being highly tuned to the express and narrow requirements identified below. We will usually accept such implementations, but they are subject to being classified as Stripped.

Requirements

The recommended URI is /plaintext.
The response content type must be set to text/plain.
The response body must be Hello, World!.
This test is not intended to exercise the allocation of memory or instantiation of objects. Therefore it is acceptable but not required to re-use a single buffer for the response text (Hello, World). However, the response must be fully composed from the response text and response headers within the scope of each request and it is not acceptable to store the entire payload of the response, or an unnaturally large subset of the response, headers inclusive, as a pre-rendered buffer. "Buffer" here refers to a byte array, byte buffer, character array, character buffer, string, or string-like data structure. The spirit of the test is to require the construction of the HTTP response as is typically done by a framework or platform via concatenation of strings or similar. For example, pre-rendering a buffer with HTTP/1.1 200 OK<cr>Content-length: 15<cr>Server: Example<cr> would not be acceptable.
The response headers must include either Content-Length or Transfer-Encoding.
The response headers must include Server and Date.
gzip compression is not permitted.
Server support for HTTP Keep-Alive is strongly encouraged but not required.
Server support for HTTP/1.1 pipelining is assumed. Servers that do not support pipelining may be included but should downgrade gracefully. If you are unsure about your server's behavior with pipelining, test with the wrk load generation tool used in our tests.
If HTTP Keep-Alive is enabled, no maximum Keep-Alive timeout is specified by this test.
The request handler will be exercised at 256, 1024, 4096, and 16,384 concurrency.
The request handler will be exercised using GET requests.

Example request

GET /plaintext HTTP/1.1 Host: server User-Agent: Mozilla/5.0 (X11; Linux x86_64) Gecko/20130501 Firefox/30.0 AppleWebKit/600.00 Chrome/30.0.0000.0 Trident/10.0 Safari/600.00 Cookie: uid=12345678901234567890; __utma=1.1234567890.1234567890.1234567890.1234567890.12; wd=2560x1600 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-US,en;q=0.5 Connection: keep-alive

Example response

HTTP/1.1 200 OK Content-Length: 15 Content-Type: text/plain; charset=UTF-8 Server: Example Date: Wed, 17 Apr 2013 12:00:00 GMT Hello, World!

Web Framework Benchmarks

JSON serialization

Requirements summary

Requirements summary

Requirements summary

Requirements summary

Requirements summary

Requirements summary

Comments

Framework details

Results testing

Limitations

Introduction

Results

Making improvements

What's to come

Additional resources

Current and Previous Rounds

Unofficial Results

Motivation

Terminology

Expected questions

Frameworks and configuration

The tests

Environment

Results

Contributions

Join the conversation

Simulating production environments

Environment details

Cloud environments

Physical hardware environments

General resources

Test requirements

General requirements

Test type 1: JSON serialization

Requirements

Example request

Example response

Test type 2: Single database query

Requirements

Example request

Example response

Test type 3: Multiple database queries

Requirements

Example request

Example response

Test type 4: Fortunes

Requirements

Example request

Example response

Minimum template

Test type 5: Database updates

Requirements

Example request

Example response

Test type 6: Plaintext

Requirements

Example request

Example response